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PREFACE 



In September of 1967, the author became the director of the 
Health Services Mobility Study, a project funded by the Office of Eco- 
nomic Opportunity* The grant carried the charge that the project-in- 
vestigate the impediments to upward occupational mobility in New York 
City Municipal Hospitals and that it suggest means of overcoming ob- 
stacles to such mobility. It was a one-year grant. 

Ten years later, the Health Servi-ss Mobility Study (HSMS) is 
ending its research and development activiti3s* During that time, HSMS 
examined the occupational structure of New York City Municipal Hospitals 
and investigated the problems of skill shortages and credentialing^l It 
then undertook to design a method to promote occupational mobility by 
tying job requirements to curriculum design "^n a single system, 

HSMS developed, field tested, and applied a new task analysis 
method to analyze work and design job ladders. It produced a method of 
curriculum design using task data that also makes it possible to design 
educational ladders to parallel job ladders. The HSMS method can be 
used to make job structures and curricula responsive to quality standards 
and the needs of consumers, 

HSMS has made theoretical contributions to the fields of job 
analysis, curriculum development, and occupational testing. It has 
helped to promote the concepts of upward occupational and educational 
mobility, and has developed a design for a safe practice, quality assur- 
ance program in diagnostic radiology. 

The HSMS method was pilot-tested in an ambulatory care com- 
munity health center. It was given a full-scale application in diag- 
nostic radiology. An abbreviated version of the method was applied to 
the technologist, technician and aide functions in radiation therapy 
and diagnostic ultrasound, A curriculum has been developed covering 
the aide, technician, and technologist levels in diagnostic radiology. 

Although these applications have been in health services 
occupations, all of the components of the method are generic and can 
be applied to any work activity in any industry. 



Eleanor Gilpatrick and Paul Corliss, The Occupational Structure of New 
York City Municipal Hospitals , New York: Health Services Mobility 
Study and/or Praeger Publishers (Research Report No, 2), 1970, 
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Now the time has come to share the method so that it can be 
used by others. This research report offers all the components of the 
HSMS method of task analysis, job ladder design, and curriculum devel- 
opment for use as a system or in part. It is offered to any institu- 
tion that wishes to expend time and resources to rationally structure 
work, utilize its labor force, evaluate its work performance, develop 
job ladders, design job-related education, or create work-related test 
instruments. This material is reported as follows: 



Research THE HEALTH SERVICES MOBILITY STUDY METHOD 

Rpt. No. 11 OF TASK ANALYSIS AND CURRICULUM DESIGN. 

Vol. 1 Basic Tools: The Concepts, Task Identification, 

Skill Scales and Knowledge System. 

Vol. 2 Writing Task Descriptions and Scaling 

Tasks for Skills and Knowledge: A Manual. 

(Also contains an abbreviated version of the task 
description method.) 

Vol. 3 Using the Computer to Develop Job Ladders. 

(Includes technical material, computer programs, 
scholarly review, and a mini-manual for performance 
evaluation. ) 

Vol. 4 Developing Curriculum Objectives from Task 

Data : A Manual . 



The reader is directed to other HSMS documents for additional 
information not contained in Research Report No. 11 as follows: 



Technical 
Rpt. No. 11 



Working 
Paper No. 



HEALTH SERVICES MOBILITY STUDY: FINAL REPORT 
FOR THE PERIOD OCTOBER 1967 THROUGH MARCH 1972. 

(Contains a review of the literature in task analysis 
and the derivation of the HSMS task analysis method.) 

THE DESIGN OF CURRICULUM GUIDELINES FOR EDUCATIONAL LADDERS 
USING TASK DATA. 

(Earlier version of HSMS curriculum design method. Con- 
tains a review of the literature in occupational curriculum 
design and behavioral objectives, and other related ma- 
terial . ) 



Research TASK DESCRIPTIONS IN DIAGNOSTIC RADIOLOGY. 

Rpt. No. 7 

Vol. 1 Medical Tasks: What the Radiologist Does. 

Vol. 2 Radiologic Technologist Tasks Dealing With Patient 

Procedures . 

Vol. 3 Machine-Related, Patient Care and Administrative Tasks: 

\^at Radiologists, Technologists, Nurses, and Physicists 
Do To Run Things and Look After Patients and Equipment. 

Vol. 4 Index of Tasks by Code Number and Extended Name. 



Research USING TASK DATA IN DIAGNOSTIC RADIOLOGY. 
Rpt. No. 8 

Vol. 1 Job Ladders: Assigning Tasks to Jobs. 

Vol. 2 Curriculum Objectives for Radiologic Technology. 



Research THE TECHNOLOGIST FUNCTION IN FIELDS RELATED TO RADIOLOGY: 

Rpt. No. 9 TASKS IN RADIATION THERAPY AND DIAGNOSTIC ULTRASOUND. 



Research RELATING TECHNOLOGIST TASKS IN DIAGNOSTIC 

Rpt. No. 10 RADIOLOGY, ULTRASOUND AND RADIATION THERAPY. 



Working USING TASK DATA FOR PERFORMANCE EVALUATION AND 

Paper No. 12 PROFICIENCY TESTING. (tentative title) 

(Theory of criterion-referenced and norm-referenced 
testing; use of task data as inputs to testing. The 
HSMS theoretical document on occupational proficiency 
tests and issues of validity.) 
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CHAPTER 1 

HSMS STATISTICAL ANALYSIS: INTRODUCTION 

This third volume of Research Report No, 11 is a manual for 
using HSMS task data to design job structures and job ladders; it also 
describes the use of task data for institutional performance evaluation 
and manpower planning. 

The four volumes of this report present the entire HSMS task 
analysis and curriculum design system. Volume 1 contains the HSMS skill 
and knowledge scales and the HSMS Knowledge Classification System, It 
is the companion document to Volume 2, which describes the work carried 
out by the director of a task analysis project and its job analysts and 
covers task identification, task description, skill scaling, and knowl- 
edge identification and scaling. Volume 4 presents the HSMS curriculum 
design method, 

ABOUT VOLUME 3 

This volume describes how HSMS uses computer-based statisti- 
cal procedures to design job structures and job ladders. It presents 
the HSMS computer programs and describes hov to use them to group tasks, 
arrange them into jobs and job ladders, and how to use the results for 
performance evaluation and manpower planning. 

Chapter 1 is a manual for coding and preparing the HSMS task 
data for computer-based analysis. Chapter 2 describes how to use HSMS 
task data and computer programs to group tasks into interrelated fami- 

.9^. 1-11 
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lies and hierarchies of tasks. Chapter 3 describes how to assign tasks 
to job levels, job structures, and job ladders. 

Chapter 4 indicates how the analytic results can be used by 
*an individual institution or department to make rational use of man- 
power, restructure johr, , assign tasks to jobs and titles at various 
levels, provide upward mobility, and/or evaluate task performance. 

The five HSMS computer programs are presented in Appendixes 
B through F, along with instructions for their use. Appendix A pre- 
sents a brief history of the HSMS analytic instruments, covering data 
on the definition of task, the HSMS scales, and the Knowledge Classi- 
fication System. Appendix C is a scholars' review of the HSMS method. 

CODING AND KEYPUN CHING 

l^en the HSMS tasks have been identified, described, scaled 
for skill and knowledge requirements, reviewed, and approved, the final 
data forms are ready for coding and data processing. The task identi- 
fication data appear on HSMS Task Description Sheets or Task Identifi- 
cation Summary Sheets. The skill scale data appear on HSMS Skill Scal- 
ing Sheets, and the knowledge category and scale data appear on HSMS 
Knowledge Identification Sheets. The data must then be transferred to 
punched computer cards- This is done by use of code sheets which re- 
present the columns on computer cards. This section describes the code 
sheets and how they are used. 
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Overview 

Each task which is to enter the data base is a basic unit 
for purposes of statistical analysis. Each is uniquely identified by 
its Code Number. All data cards for a given task must include the 
task^s Code Number. 

To facilitate information retrieval, we made it possible 
to identify a task by several criteria such as job title, institution, 
department, task frequency, and an abbreviated name. The latter makes 
it possible to easily interpret data listings and the analytic results 
which appear on computer print-outs. Each data card contains a fixed 
number of columns set aside for identification purposes; but only the 
first card includes the abbreviated task name. 

The skill scale data posea no special problem. With sixteen 
skill scale values to record for each task, we decided that the second 
data card for a ::ask would have a fixed format in which a specific two- 
column field would be assigned to each skill scale, and scale values 
would be punched excluding the decimal points. Thus, a task's scale 
value for a skill can be determined by the numbers punched in given 
columns on the card. 

In the case of the knowledge categories, no such fixed as- 
signment of category to column could be made because of the vast num- 
ber of knowledge categories in the System. Even if we could know in 
advance which categories would be involved in each "run," a fixed col- 

1-3 
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umn assignment for each category would require hundreds of cards and 
thousands of zero punches, since most categories are not needed for 
most tasks, A new format would be needed for each run. This problem 
was solved by the decision to use a format for knowledge data in 
which eight columns are assigned for entry of a knowledge category's 
code number, followed by two columns in which to enter the scale value. 
For any given task there as many knowledge data cards as are needed to 
cover all the knowledge categories it requires at non-zero values. 
Each task's first data card tells the computer how many other data 
cards to expect for the given task. 

Thus, any task must have at least two data cards. Tasks 
that require no knowledge categories above zero on the knowledge scale 
have only two cards. All other tasks have as many data cards as are 
required to record all the scale value data. The data cards for each 
task are numbered TOO, TOl, T02, and so on; each contains the Task Code 
Number, any optional identif icatin^ Information selected, and the data 
that the particular card has been designed to carry. 

Figure 1 presents the HSMS TOO code sheet for a task. There 
are four banks which together represent the 80 columns of a computer 
card. The first line of each bank gives the column numbers. The second 
line indicates the column designations, i.e,, the instructions to the 
coder on what to enter on the third line. The third line is what the 
keypunch operator punches, based on what has been entered by the coder. 

Columns 1, 2, and 3 represent the card number for the given 
task. A * is always punched in column 1 to designate (T)ask data, 

1-4 




Figure 1. HSMS CARD TOO SHEET; SUMMARY CARD 
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Indicates blank. Coded by; ^ Checked by 



This distinguishes the task data cards from other cards. Columns 5 
through 27 are identical for a given task on all its cards. The 
task*s Code ( Iden tificati on) Number is a lways pu nched in colunms^ 5 
through 10 , right justified in the field. Columns 12 through 27 con- 
tain optional data which are not needed for statistical analysis. HSMS 
enters an "N" in column 25 to designate a task that has been reviewed 
and is now in "normative" final form. 

Columns 29 to 80 are unique to card TOO. Columns 29 and 30 
are coded after all the skill and knowledge scale values have been 
coded. The number entered tells the computer how many cards are to 
follow for the task. The figure corresponds to the number of th^ last 
knowledge card for the task. 

Figure 2 is the code sheet for the TOl card; it is designed 
to include all the skill scale data for a task. There are column de- 
signations for all 16 skill scales. If a task has been scaled at zero 
for a skill, zeroes should be codec and punched in the columns ass igned 
to the skill . 

Figure 3 is the code sheet for all the knowledge data cards 
for the tasks. Columns 2 and 3 are coded in numerical order as the 
code sheets are filled out, beginning with T02. There is room for data 
for four knowledge categories per card. The coder enters a knowledge 
category's own 8-digit code number; this is followed by the nonzero 
scale value for the category (omitting the decimal point) in the two 
columns designated. 
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Figure 2* 



HSMS CARD TQl SHEET; SKILL SCALES 
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Figure 3* HSMS CARD T02 (Or Higher) SHEET: KNOWLEDGE 




(X) Indicates blank. 

er|c vj 



Coded by: 
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The knowledge category columns mus t be filled out in order 
with no blank fields left until after the last knowledge category . This 
is because the HSMS "MATRIX" program reads a blank knowledge field as 
the end of the information for the particular task."^ 

The code sheets must be carefully checked at least once by 
someone other than the original coder. This is especially important in 
the case of the 8-digit knowl^?dge category codes. 

The Use of A Code Book 

The code sheets are designed to be self-explanatory. A 
coder should be able to fill out each sheet by referring to a task's 
data sheets. However, the organization carrying out task analysis may 
wish to maintain a Code Book to cover the task data collected. In ad- 
dition to a listing of task names and code numbers, the user may wish 
to code job titles, performer, J-^nartments, and/or shifts. If the user 
is part of a central office or a consortium arrangement, it may be nec- 
essary to give each individual institution a code number. 

The following are rules for assigning code numbers : 

1, Code numbers are assigned in consecu tive order as 
new information is entered. 

2. Each task, regardless of performer or institution, 
will have been assigned a unique Ta sk Code Number. 
All overlap tasks should have the same Task Code 
Number. (The data for a given task enter HSMS sta- 

This means that if any categories are to be deleted after keypunching, 
this cannot be done merely by er'asin^'; the data. Any category that is 
eliminated must be replaced. It is easiest to replace the eliminated 
category by the last one entered on the last card, and then erasing 
the last entry. If an entire card is eliminated, the count in columns 
29-30 on card TOO must be changed. 
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tistical analy«^*s only once; i.e., a code number 
is included only once.) 



Columns 



Each job title, regardless of department or institu- 
tion, is assigned a unique job title code number . 
All titles which are the same will have the sane 
title code number. 

Each institution is assigned a unique institution 
code number . 

Each department name, regardless of institution, is 
assigned a unique department code number . All de- 
partment names which are the same will have the 
same department code number. 

Performers are assigned code numbers separately for 
each institution, starting from 001 in each case. 
A performer's code is preceded by the institution's 
code, and so a unique number results. 

Coding for all cards: Identification Information. 



Code 



Instructions 



T Always a T to designate task data. 

00 to 99 Card number as appropriate. 

000001 to Task Code (Identification) Number. Right 
999999 justify in field. 

001 to 999 Job Title Code. Optional. Right justify. 

1 to 9 Institutional Code. Optional. 

001 to 999 Performer Code. Optional. Right justify. 

01 to 99 Department Code. Optional. Right justify, 

1 to 9 Shift Code. Optional. 

Type of task as per designations used for 
analysis. HSMS uses "N" for normative 
tasks . Optional. 

Frequency scale value for given performer 
or job tide. Optional. 

.Special designation. Optional. 
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8. Coding unique to Card TOO: Identification of task and card* 



Columns Code Instructions 



29-30 01 to 99 Number of cards to follow ' Card TOl is 01; 

add al] the cards needed for all the knowl- 
edge categories (4 to a card), from T02, on. 
Same as columns 2-3 on the task^s last card. 
Right justify. 

32-78 — Abbreviated name of the task. Use words and 

abbreviations that unambiguously refer to 
the task^s abbreviated task name on the Task 
Description Sheet or Summary Sheet. 



9. Coding unique to Card TOl: Skill Scales. 



Columns Skill Scale 



32- 


33 


Scale 


value 


for 


Locomotion (Scale 2). 




35- 


36 


Scale 


value 


for 


Object Manipulation (Scale 3). 




38- 


39 


Scale 


value 


for 


Guiding or Steering (Scale 4). 








bcaie 


value 


1 or 


Human Interaction (Scale 5). 




44- 


■45 


Scale 


value 


for 


Leadership (Scale 6). 




47- 


■48 


Scale 


value 


for 


Oral Use of a Relevant Language (Scale 


7). 


50- 


■51 


Scale 


value 


for 


Reading Use of a Relevant Language (Scale 8). 


53- 


■54 


Scale 


value 


for 


Written use of a Relevant Language (Scale 9). 


56- 


■57 


Scale 


value 


for 


Decision Making on Methods (Scal(^ 10). 




59- 


■60 


Scale 


value 


for 


Decision Making on Quality (Scale 11). 




62- 


■63 


Scale 


value 


for 


Figural Skills (Scale 12). 




65- 


■66 


Scale 


value 


for 


Symbolic Skills (Scale 13). 




68- 


-69 


Scale 


value 


for 


Taxonomic Skills (Scale 14). 




71- 


-72 


Scale 


value 


for 


Implicative Skills (Scale 15). 




74- 


■75 


Scale 


value 


for 


Financial Consequences of Error (Scale 


16). 


77- 


-78 


Scale 


value 


for 


Consequences of Error to Humans (Scale 


17). 



Code scale value for each skill scale without decimal point in the two- 
column field designated. Code all zeroes; code a zero scale value as 00. 



10. Coding unique to Card T02 and Higher: Knowledge Data, 



Columns Code ^ Instructions 



30-37 8-digit code First Knowledge Classification System cate- 

gory code. 

39^40 Nonzero scale value for Levels of Knowledge 

(Scale 18) for preceding category, without 
decimal • 

43-50 8-digit code Next Knowledge Classification System cate- 

gory code. 

52-53 Nonzero scale value for Levels of Knowledge 

(Scale 18) for preceding category, without 
decimal • 

56-63 8-digit code Next Knowledge Classification System cate- 

gory code. 

65»56 Nonzero scale value for Levels of Knowledge 

(Scale 15) for preceding category, without 
decimal . 

69-76 8-digit code Next Knowledge Classification System cate- 

gory code. 

78-79 Nonzero scale value for Levels of Knowledge 

(Scale 18) for preceding category, without 
decimal. 



Keypunching 

It is most efficient to arrange the code sheets in three groups 

1. A set of TOO sheets in numerical order by Task Code 
Number. 

2. A set of TOl sheets in numstical order by Task Code 
Number. 

3. A set of T02 and higher sheets in numerical order by 
Task Code Number, and within that, in numerical order 
by Code Number. 
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This arrangement makes it possible to use drum set-ups for 
keypunching, and provides listings which can be inspected visually for 
illegal punches in blank fields • The cards should be keypunched, veri- 
fied, and listed. They should then be proofed against the original data 
sheets > This provides a double check against errors picked up in coding* 

Once this is done, the task data card decks should be struc- 
tured so that the cards appear in numerical order by Task Code Number, 
and within each task, in numerical card number order. At this point the 
task data are ready for submission to th>i computer for their first com- 
puter checks and analyses. 

USE OF THE "EDIT" PROGRAM TO 
CHECK AND SELECT DATA 

HSMS employes a multi-purpose computer program, "EDIT," which 
prepares the task data for use with its other analytic and statistical 
programs. "EDIT" is first used to check that the data cards arc arranged 
properly for submission; it also performs several error checks. We 
generally submit all the data cards to a check by EDIT before putting 
the data on magnetic tape in permanent form. 

Checks 

We have EDIT carry out a series of checks in the first com- 
puter run with a set of data by selecting the proper options in the 
EDIT program. The following are the checks carried out: 



3 

For a detailed description of EDIT, instructions for use, and a listing 
of the program, see Appendix A. 



1, A check that each task (task Code Number) appears 
only once in the set of daca, 

2, A check that the number of cards for a task is con- 
sistent with the number indicated on its card TOO, 

3, A check that al) the cards for a task have the same 
Task Code Number* 

4, A check that data cards appear in proper sequence 
(as indicated in columns 1-3). 

5, A check that punches occur only in permissible columns. 

6, A check that scale values all end in 0 or 5. 

?• A check that all knowledge categories have a scale 
value above 00. 

8. A check that knowledge categories appear only once 
for a given task. 

The first EDIT submission is usually run tu include an op- 
tional listing of the data cards. Thus, when there is an error message 
relating to the checks, or if the analyst finds an error, it is possible 
to refer to the data listing to find the problem. Having the data in 
card image form makes it relatively easy to plan and make corrections. 

One of the functions of EDIT is to create a matrix of tasks 
by skill and knowledge categories; EDIT enters zeroes in this matrix 
whenever a category in the data ^ose is not required b- a task. 

EDIT then orders the data base in successive matrix arrange- 
ments and lists the resulting information in various predetermined for- 
mats. In the first run with a set of data, some cf the format listings 
are used by staff to check for "legal" and correct vie of knowledge 
categories. The four format listings of data are as follows: 
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1. EDIT gives each skill and knowledge category an in- 
ternal number. These are listed in the order in 
which they appear in the input (data card) file, to- 
gether with the number of tasks in which each appears 
(frequency), and the Task Code Number and scale value 
of each task in which the variable is scaled above 00, 

2. EDIT lists the information described in 1, above, in 
descending order of the skill or knowledge category 
frequency of occurrence in the tasks ♦ The original 
internal numbers, the Task Code Numbers, and the 
scale values appear again in order of frequency, 

3, EDIT again lists the skill and knowledge information 
described above, in order by skills first (in a pre- 
set order), followed by the knowledge categories in 
ascending numerical order of their 8-digit code num- 
bers. The original internal numbers, frequency. Task 
Code Numbers and scale values appear again in this 
third order of presentation, 

4, EDIT renumbers the skill and knowledge categories 
(variables) internally and lists these in order as 
in the third listing, with skills first, followed by 
the knowledge categories in ascending order of their 
8-digit code numbers. The fourth listing omits the 
Task Code Numbers and scale values. 



The fourth listing, described above, is then used as follows 



1, The person doing the checking should be someone fami- 
liar with the data and the knowledge identification 
and scaling, 

2, The fourth listing is fairly compressed. The person 
who makes the check sets up a three columr table by 
using this print-out. The skill codes and the knowl- 
edge category codes of the print-out become the left- 
hand column of the table. 

The checker then refers to the Knowledge Classifica- 
tion System and enters the abbreviated name of the 
category next to its code number in the right-hand 
column. The third column is the listing of each 
category's frequency of occurence across tasks, 

3, A checker who has worked with the data can usually 
spot any inappropriace categories caused by coding 
or keypunching errors. When an error is found, the 
checker uses the third listing to find the tasks in 
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which the error appears. Then the data card listings 
are used to find the data errors and plan for the 
corrections , 

4, An error is usually remedied by correcting a cate- 
gory code number. If an illegal number has been used 
it may have to be eliminated entirely. The user 
should remember that, in making corrections, an il- 
legal knowledge category code- and scale value cannot 
be removed by leaving blank fields (except for the 
last knowledge category for a task). Each must be 
replaced , See footnote 3 , 

5- After the errors have been corrected the EDIT run is 
repeated and the data are entered on magnetic tape. 

Using EDIT to Select Variables 

Once the data are corrected, the EDIT program is used again 
to arrange the raw data matrix for further analysis. In the data matrix 
created, the rows are the tasks; the columns are the skill and knowledge 
variables; and the entries are the scale values. The HSMS purpose is to 
go from this basic data matrix to the assignment of tasks to job levels 
and to recommendations on job ladders. The essential problem is to 
group a lai^3 number of tasks that require a large number of skill and 
knowledge variables at varying scale values into a meaningful smaller 
number of groupings so that the underlying association of skills and 
knowledges (variance) will be reflected in the groupings of the tasks. 

To solve the problem HSMS employs a ^'orm of factor analysis 

called "principal components analysis*' in a procedure we call "two-mode" 

factor analysis. It is used to assign tasks to groupings that require 

related skills and knowledges. The word "factor" means grouping, (The 

way skills and knowledge categories group together in factors determines 

the way tasks can be grouped together in factors,) 
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The HSMS method requires the use of several computer programs. 
Before they can be used it is necessary to make sure that each particu- 
lar "run** with its tasks and skill and knowledge variables conforms to 
general statistical requirements, i.e., that there are substantially 
more tasks than variables, that variables with very low frequencies 
across tasks are eliminated, and that there are no more than 144 vari- 
ables (the current limit of the HSMS "two-mode" programs). This re- 
quires selection. 

The selection stage is composed of two parts: 

1. Selection of the tasks to enter the statistical ana- 
lysis for given computer "runs." 

2. Selection of the skill and knowledge variables that 
have sufficient frequency to enter into the factor 
analysis for each run. 

The first selection step is the choice of tasks for ana- 
lysis. The user may be interested in the interrelationships among a 
large number of tasks, such as those covering one or more departments 
or services, and may also wish to examine the factor structure of a 
subset of tasks. If this is the case, the analysis is carried out sep- 
arately for each set of data; when the input decks are structured for 
computer submissions the "runs" are given separate designations to 
avoid confusion. 

After the ertor check, EDIT is used by itself to obtain cor- 
rected listings for each "^nn." The listings are then used to aid the 
user in deciding which variables to eliminate so that the number of 
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skill and knolwedge variables is an acceptable quantity, based on pro- 
gram limits and/or statistical requi>-ements . 

The user works with the second listing, described earlier, 
which lists the variables in descendir . order of frequency. The table 
used for checking, described earlier, can be redone and used as a ref- 
erence so that the user can consider the identity of the knowledge cate- 
gories (which are listed by code number). 

Working with the second EuIT listing, the user numbers the - 
skill and knowledge variables from 1 to 144, in descending order of fre- 
' quency and notes the frequency of the 144th variable. Generally, vari- 
ables with lower frequencies- are eliminated to bring the total to 144. 

Even if the original number of variables is 144 or less, it 
is nonetheless advisable for statistical reasons to use a frequency 
cutoff of at least 4. Retaining variables with frequencies lower than 
5 distorts the results as a consequence of the high number of iit. 
scale values that appear in the raw data matrix. HSMS selects a mini- 
mum cutoff option of 4; the total number of variables is then determined 
by the number of variables retained after the cutoff is used. 

If the 144th variable falls at a relatively high frequency, 
and the selection of this frequency as a cutoff figure will eliminate 
more variables than the number needed to arrive at 144, it is desirable 
to select a lower frequency cutoff. With large numbers of tasks in 
run it is desirable to retain a full set of 144 variables when these 
all have relatively high frequencies. 
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HSMS usually selects a cutoff figure which results in several 



more than lAA variables; we then selectively eliminate additional vari- 
ables to bring the total to lAA, We may eliminate either broad or fine- 
level categories if both are identified for a given run. If the tech- 
nologist level is of particular interest, we may eliminate some categor- 
ies found only at the professional level. The selective elimination of 
variables permits the user to retain relevant data based on the needs 
of the particular analysis. 

We record the cutoff and deletion decisions for each run, 
EDIT is then run with the statistical programs, as described in the 
next chapter. The EDIT listings described earlier are repeated in 
the subsequent print-outs, and have the following characteristics: 



1, The fourth EDIT listing presents the variables se- 
lected; they are renumbered internally and listed in 
order with the skills first, followed by the knowl- 
edge categories in ascending order of their 8-digit 
code numbers. Frequency data are listed, but not 
task and scale value data. When DELETE and CUTOFF 
options have been selected, this listing does not 
include the eliminated categories,* (In the three 
prior li<5tings the categories eliminated as a result 
of DELETE or CUTOFF options are listed and marked 
by asterisks,) This listing serves as a "Variable 
Description Dictionary," 



DELETE and CUTOFF are the EDIT options used to reduce the number of 
skill and knowledge variables which are copied to tape, DELETE is 
used to name specific skill and knowledge categories to be eliminated; 
CUTOFF is used to specify the frequency at or below which variables 
are automatically eliminated. 

Eliminating variables with low frequencies for factor analysis does not 
mean the loss of information about such variables. Later in the analy- 
sis, when tasks are arranged in order of difficulty for assignment to 
job levels, and in the design of curricula, all the data are dealt with. 
At this stage variables of low frequency do not provide information for 
grouping tasks, and their elimination is not a loss, 
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2. EDIT provides a "Task Description Dictionary" which 
lists each task^s internal (computer) number in nu- 
merical order and its Task Code Number, together with 
the abbreviated name of the task, as found on card TOO 
for the task. 

3. The final (internal) numbers assigned to skill and 
knowledge variables and to tasks are the numerical 
references for observations (tasks) and variables 
(skill and knowledge categories) when PCVARIM, X2M0FA, 
and X2MFA2 (the HSMS statistical programs) are used. 



Using EDIT to Prepare Variables for Analysis 

When EDIT is run with the statistical programs, one additional 
option is used to prepare the variables for statistical analysis. The 
EDIT program performs a logarithmic transformation on the data to bring 
them to a closer approximation of linearity among variables.^ 

Thus, at this stage, there is a reduced data matrix for each 
"run" whose rows are the tasks selected for the given run, and whose 
columns are the 144 or less variables selected for that run; the entries 
are the scale values adjusted by EDIT. 



The NORMALIZE option permits a logarithmic transformation of the data 
to adjust for a large number of zeroes in the data base. (Tasks which 
do not require knowledge categories required by any other task are 
scaled at zero.) The NORMALIZE option follows the formula: 

X ^ SQRT (X + 0 5). where X is a scale value. 
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CHAPTER 2 



ANALYTIC TECHNIQUES FOR GROUPING TASKS 

This chapter describes the use of three computer programs to 
group HSMS tasks into interrelated families of tasks based on the skill 
and knowledge scale data* The first section presents a general discus- 
sion of the HSMS method; the second section describes the use of the 
HSMS FCVARIM program for grouping variables; and the third section de- 
scribes the use of the "two-mode" factor analysis programs for grouping 
tasks. The two latter sections serve as a method manual. Details about 
the three programs, instructions for computer submissions, and program 
listings appear in Appendixes C, D, and E. 

OVERVIEW 

111 order to arrive at a grouping of tasks, the HSMS method 
uses statistical procedures and techniques which are also used by scien- 
tists engaged in building predictive models* However^ the HSMS use of 
these analytic techniques is a descriptive, applied use of statistics* 
The techniques serve only to organize and simplify the data; the results 
are suggestive, not predictive. The overall method for grouping tasks 
is as follows: 

1. The basic data are the skill and knowledge scale 
values assigned to tasks. The "variables" are the 
skills and the knowledge categori(is. The "obser- 
vations" or "subjects" are the tasks. 

2. Principal component factor analysis is used to ex- 
amine the factor structure (relationships) among 
the variables. A solution (number of factors) is 
selected which best describes the relationships 
among variables. This becomes the basis for 
grouping tasks . 
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3. "Two-mode" factor analysis is used to examine the 

factor structure of tasks based on the factor struc- 
ture of variables and to assign each task to a factor. 

4* The factors are named for the skill, knowledge, and 
work content that characterizes the variables or tasks 
that are assigned to factors. 

5. The task factors are the basis for structuring jobs, 
designing job ladders, and designing curricula, be- 
cause the^ represent groupings of tasks which have 
much in common with respect to skill and knowledge 
requirements . 

Factor analysis is an analytic technique that is used when 
the statistical relationships among a large number of variables are of 
interest. The object is to replace the separate relationships of each 
variable with every other variable with a smaller number of interrelated 
variable groups (factors). Each factor is essentially a construct that 
expresses the interrelationships within a particular group. 

The initial factor analysis technique used by HSMS is called 
"principal components analysis." It examines the correlation of each 
variable with every other variable across a given set of observations 
(tasks) and groups these to best account for all the variability among 
the variables. Using various criteria, the analyst selects a factor 
solution, i.e., the number of factors in which to group the variables. 
A factor accounts for the variance among several variables in a test 
space that is analagous to the way a regression line accounts for the 
variance between two variables in two-dimensional space. 

The factor analysis program which provides the solutions for 
grouping the HSMS variables is called PCVARTM, an abbreviation for Prin- 
cipal Components Factor Analysis with Varimax Rotation. The "two-mode" 
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factor analysis programs which produce the HSMS "task factors" are 
called Two-Mode Factor Analysis Part One (X2M0FA) and Part Two (X2MFA2)» 

To arrive at principal axis (PA) factors, the PCVARIM pro- 
gram and the two-mode programs use a principal components technique, with 
unities in the diagonal rather than communality estimates. HSMS uses 
a correlation matrix of variables (rather than a covariance or cross- 
products matrix), and varimax rotation of the PA factors, which produces 
an orthogonal (rather than oblique) factor solution."^ 

Unlike other factor analytic techniques which first reduce 
the total variability in a test space, the principal components tech- 
nique summarizes the total variability in a test space into a smaller 
number of orthogonal components. The factors produced are maximally 
independent of one another. 

In the HSMS two-mode programs the reduced data matrix is 
used to form two conceptually different but necessarily related corre- 
lation matrixes. One is the correlation of every variable with every 
other variable across all the tasks (as in the PCVARIM program); the 
other is the correlation of every task with every other task across 
all the variables. 

The two-mode programs produce the principal components of 
the variable matrix as in the PCVARIM solution; in addition, the 

The use of correlation matrixes provides solutions that are not depend- 
ent on the standard deviations of the variables, as covariance matrixes 
are, nor on the mean? ard standard deviations, as cross-product matrixes 
are. 
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programs produce a transfer of the principal components of the task 

correlation matrix based on the Eckert-Young theorem. This produces 

2 

the task factors which reflect the variable i.actoJs. 
SELECTING THE VARIABLE FACTORS 

HSMS uses the PCVARIM program to select the "solution" (num- 
ber of factors) which best groups the skill and knowledge variables. 
PCVARIM is used with EDIT, as described in Appendix C EDIT selects 
and lists the variables in the order to be used in the PCVARIM program, 
and "normalizes" the data. 

PCVARIM makes it possible to examine a variety of factor so- 
lutions. Solutions with as many as twelve factors or as few as two fac- 
tors may be examined, as well as all the others in between. An EDIT 
print-out can be ordered. The PCVARIM pvint-out includes means and 
standard deviations, the correlation matrix, and other information such 
as Eigenvalues, principal axi.s factors, and communalities . 

Of chief interest are the "Varimax Factor Loadings." These 
are presented in arrays, starting with the largest factor solution re- 
quested, and ending with the solution containing the smallest number of 
factors requested. The user examines each factor solution and decides 
on the one that seems most appropriate. 
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After the "variable mode" is rotated to simple structure, the "task 
mode" is "counter rotated" by obtaining the transformed characteristic 
vectors of the observation mode induced by the varimax rotation of the 
variable mode following the Eckert-Young Theorem. See Appendix G for 
a discussion of the procedures and Appendixes D ^nd E for the programs < 
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For each ''solution" requested there is an array arranged as 
a matrix, in which the columns are numbered and stand for the factors 
in the given solution. The rows are numbered and stand for the vari- 
ables, in the order determined by EDIT, At 'the bottom of each array 
are data on the variance accounted for by each factor. 

The entries in the arrays are the varimax factor loadings. 
These are decimal numbers, of which none can be greater than ,999. 
Every variable has a loading on every factor in a solution. Variables 
can load on factors within the range of ±.999, Variables which are 
positively interrelated on a factor will have the same sign. The + or 
- sign has no other intrinsic meaning, A loading of 1,400 or more is 
of interest. For a four-factor solution, the first five variables and 
Che end of the array might appear as follows: 



VAR. VARIMAX FACTOR LOADINGS 

NO, 1 2 3 A 

1 ,429 X -257 -.009 -.246 

2 ,578 X .139 -.622 x ,039 

3 -635 X ,360 -.218 -.113 

4 -,054 -,034 • ,023 -.220 

5 -,006 -,072 -.909 x .015 
etc, etc, etc, etc, etc, 

VARIANCE 18,207 10,037 8,547 5,876 

PCT VAR .207 .114 .097 .067 

CUM PCT .207 .321 ,418 .485 



Note: An x denotes a high-loading variable and is entered by the 
analyst during inspection of the print-out. 



The loadiiigs reflect the extent to which a variable's scale 
value variations contribute to the variance accounted for by the fac- 
tor in the given solution. In examining several factor solutions (for 
example, ten factors, nine factors, and so on, down to three factors), 
the analyst notes which variables have "high loadings" (.450 or higher, 
independent of sign) on each factor in each solution* These variables 
"determine" a factor; their identities give some sense of the underly- 
ing meaning of the factor* In the case of HSMS data, a factor may sug- 
gest the skills and knowledge categories needed for a work function, a 
specialty, a type of service, or a type of procedure. 

The choice of an acceptable factor solution (that is, the 
choice of five factors rather than four or six factors) can be based 
on statistical criteria, on common sense, or on a combination of these. 

HSMS eliminates solutions which have any factors with less 
than three variables with relatively high loadings (i.e., 1.450 or 
more), and solutions in which no underlying structure of interest is 
evident. The choice is limited to solutions wMch make sense, those 
which account for most variables, and in which few variables have high 
loadings on several factors. We look for stability of factor structrres 
across several factor solutions, and choose that solution whose underly- 
ing structure is most easily understood in terms of what we know about 
the nature of ^he work being studied. 

The specific analytic work uith the PCVARIM output is as 

follows: 
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For each factor solution array, the analyst places 
a check mark, x, or some other indicator next to 
each loading at i.450 or higher. 

The analyst eliminates all solutions in which one 
or more factors (columns) have no loadings ot 
±.450 or higher. 

The analyst prepares analysis sheets for the remain- 
ing solutions : 

a. For each factor in a solution the analyst lists 
the variables that load at ±.450 or higher. This 
is done by listing the internal nut^.ber of the 
variable as shown in the array and then writing 
in the name of the skill or the code number and 
the name of the knowledge category. 

b. The name of a skill or knowledge category is ob- 
tained by translating the PCVARIM variable number 
into the skill code 'or knowledge number using 

the fourth EDIT listing: "Final Correspondence 
of Variable Numbers to Variable Codes as Written 
on TAPE9." The Knowledge Classification System 
name is then entered next to its code number. 

c. A single master dictionary can then be made and 
duplicated; it can be cut up and used for all 
the analysis sheets. 

The analyst examines the analysis sheets for each 
factor solution, and considers the "sense" of the 
skill and knowledge categories that load high on 
each factor in the solution. 

a. The analyst notes how the factors appear and 
are differentiated as the solutions change 
from the smallest number of factors to the 
largest number of factors being considered. 
The analyst notes what new specialties or func- 
tions are represented or expressed as new fac- 
tors appear, and notes which factors seem to 

be stable from solution to solution. 

b. The analyst considers the number of variaj)les 
that load high on several factors in a solu- 
tion and the number of variables that do not 
load high on any factors. These should both 
be at a minimum. 

c. The analyst notes any variables that load high 
with a sign opposite the one that is charac- 
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teristic of the other high-loading variables, 
A solution with such inverse loadings is dif- 
ficult to interpret, 

d. The analyst eliminates factor solutions with 
only two or three high'-loading variables in 
a factor unless some specialty of interest is 
expressed by the factor, 

5, The analyst selects the factor solution that makes 
the most sense, accounts for a substantial number 
of variables, has a minimum number of variables 
that determine more than one factor, and shows rel- 
atively stable factor structures, 

a. The Timber of factors (i,e,, the solution) is 
recorded , 

b. The characteristic sign of each factor, i,e,, 
whether the high-loading variables have plus 
signs (+) or minus signs (-) is recorded, 

c. The analyst tentatively names each factor 
based on what the association of skill and 
knowledge categories suggests with respect 
to work content or function, 

6. In reporting the factor structure of variables, 
HSMS includes loadings of ±.40 or more. High load- 
ings are reported as positive, regardless of the 
characteristic sign of the factor, except for in- 
verse loadings, which are reported as negative 
loadings. This is easier for the reader to in- 
terpret. Figure 4 presents an example of hew the 
factor structure of variables can be presentei. 



OBTAINING TASK FACTORS 

In the HSMS method, two-mode factor analysis follows after 

the selection of a skil 1-and-knowledge variable factor solution. This 

determines the number of factors and the nature of the factors to which 

tasks will be assigned. The HSMS ''two-mode" factor analysis programs 

are based oa a procedure tor factoring an individual differences matrix 

to obtain idealized subject types. The HSMS subject types are task 

groupings; the variables are the scaled skill and knowledge categories, 
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SKILLS AND KNOWLEDGES IN RADIOLOGIC TECHNOLOGY, ULTRASOUND TECHNOLOGY, RADIATION THERAPY 
TECHNOLOGY AND RELATED FUNCTIONS^: FACTOR STRUCTURE OF VARIABLES p, 1 of 5 





Factor Loadings^ 




Rad. 


Ultrasd. Radther. 


Qual.Ass. Patient 


Skill, or Knowledge Category Number and Abbreviated Name 


Tech. 


Tech. Tech. 


Materials Care 


Object Manipulation Skills 


.43 




— 


Guiding or Steering Skills 


.57 


.64 




Human Interaction Skills 








Leadership Skills 








Oral Use of a Relevant Language 


.62 






Reading Use of a Relevant Language 


.61 




.41 


Written Use of a Relevant Language 


.47 


.41 


.51 


Decision Making on Methods 


.40 




.50 


Decision Making on Quality 








Figural Skills 


.73 






Symbolic Skills 


.64 




.54 


Taxonomic Skills 


.76 






Implicative Skills 


.65 


.41 




Financial Consequences of Error 






.61 


Consequences of Error to Humans 


.67 






11731000 Normal structure and function 


.43 


.42 .56 




11731100 Regional anatomy 


.77 






11731200 Topographic anatomy 


.80 






11731300 Hematopoietic system^ 








11731400 Circulatory system 








a 

Refers to analysis ot 296 tasks listed in Table 1 by abbreviated 


task names, and 127 


skill and knowl- 



^ edge variables; 88 variables were included in factor analysis. See (d), below. 

See Table 9, Appendix D, for full names of knowledge categories and Appendix C,for skill scales, 
c Loadings of ± .40 or more are shown; blanks indicate lower loadings < 

^ Indicates category not included in factor analysis because frequency across tasks is less than 5. 
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''Part One'* of the two-mode program (X2M0FA) is run interfaced 
with EDIT, as is the case with PCVARIM. (See Appendix D.) The output of 
Part One provides card inputs for ''Part Two" (X2MFA2) . Part Two is de- 
scribed in Appendix £• 

The tasks and the skill and knowledge variables are repre- 
sented in the X2M0FA and X2MFA2 print-outs in the order assigned by 
the EDIT program. Therefore, the EDIT "dictionaries" are used to in- 
terpret the two-mode outputs with the PCVARIM outputs. 

To obtain task factors with the two-mode method we extract 
and rotate the number of variable factors chosen in the prior PCVARIM 
analysis. After counter rotation of the task mode, the task factors 
are shown in an array in which the factors are the columns and the tasks 
are the rows. The entries are the tasks' numerical loadings on the fac- 
tors. Each task has a loading on each factor. To interpret the data 
one must understand what a task factor represents, the meaning of the 
loadings, and how a task's loading on a factor arises. 

For every variable factor in the solution chosen there is a 
corresponding task factor. The skill and knowledge variables uhat de- 
termine a variable factor determine how tasks will load on the corres- 
ponding task factor. A task^s loading on a factor reflects the skills 
and knowledge categories required for the task, the scale values at 
which they are required, and the loading of those particular variables 
on the corresponding variable factor. 
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For any given task, the more skill and knowledge categories 
it requires of those variables that determine a given variable factor, 
the higher the task's loading will be on the corresponding task factor. 
The higher the task's scale values for those variables, the higher the 
task's loading on the factor. (The influence of ^ach variable can be 
estimated by noting the variable's loading on che variable factor.) 
Since a task will have some numerical loading on all task factors, a 
given task can load low or negatively (inversely) on factors that are 
determined by skill or knowledge categories not required for the task. 

The characteristic sign of the variable factor determines 
the sign of the task factor. Within this, a task's loading can be 
greater than an integer and can range anywhere down to zero, through 
zero, to the range of values of the opposite sign. For a four-factor 
solution, the first five tasks in the array might appear as follows: 



COUNTER ROTATION OF SECOND MODE 



Task 




1 (+) 


2 (+) 


3 (-) 


4 (-) 


1 




.3757 X 


-.0974 


.0326 


.0215 


2 




-.1645 


-.0751 


.0832 


.0347 X 


3 




.0057 


.4333 X 


.0287 


.0405 


A 




.0230 


.0065 


-1.0979 X 


.0082 


5 




.1692 


.0600 


-.0688 


.8189 X 


etc. 




etc . 


etc. 


etc . 


etc . 


Note: 


An X 


denotes the assignme.it of a 


task to a factor. 


Plus (-'-) and 




minus 


(-) signs at 


the top of each column denote the 


character- 




istic 


sign of the 


factor. These 


are entered by the 


analyst dur- 




ing inspection of 


the print-out. 







The higher the numerical value of a loading that has a sign op- 
posite the characteristic sign of the factor, the lower the 
value of the loading on the factor. 
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It is possible to examine a task's loading on all the fac- 
tors and assign it ( -> the factor on which it has its highest loading 
(within sign). A task loads highest on the factor with which it has 
most in common in terms of skills and knowledges. . - 

Low-level tasks which require few skills and knowledge cate- 
gories and low scale values will have low and/or inverse sign loadings 
on all factors. The differences in loadings on factors for low-level 
tasks is so insignificant, that a common-sense assignment of such tasks 
to factors is often preferable to a mechanical statistical rule for as- 
signment of tasks to factors. 

When low-level tasks such as Task 2 in the example listed 
above have ambiguous loadings , HSMS assigns each to a factor which in- 
cludes other tasks with similar requirements and with similar loadings 
in a context where the task makes sense. Generally we used a people- 
oriented factor and a materials-oriented factor for such assignments. 

The specific analytic work with the Two-Mode Part Two output 
is as follows: 

1. The print-out of Two Mode Factor Analysis Part Two 
(X2MFA2) is used in the assignment of tasks to fac- 
tors. The relevant portion of the print-out is the 

arrav entitled, "Counter Rotation of Variable 

Mode." A number is included in the space indicated 
in this heading title which corresponds to the num- 
ber of tasks in the given run. 3 



^ The words "counter rotation" and this number differentiate this array 
from the "Rotation of the Variable Mode" which corresponds to the array 
for the first mode, i.e., the skill and knowledge category mode, which 
is "rotated." For HSMS purposes, the counter rotated mode, the second 
mode, is the task mode. 
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2. The analyst obtains the "Task Description Dictionary" 
print-out which is part of the EDIT output. This 
lists the internal number of each task, each task's 
Code Number, and the abbreviated task name. These 
are cut and mounted on the print-out of the Counter 
Rotation Mode array so that the internal number 

of the task on the far left of the array corresponds 
to the internal numbering of the task in the EDIT 
dictionary. The result is a table in which one can 
read the identity of every task in the analysis and 
its loading on every task factor. 

3. The analysts enters the characteristic sign and fac- 
tor number of each factor at the top of each page 

of the array. (The number is listed only on the 
first page.) The sign is obtained by referring to 
the counterpart skill and knowledge variable factor. 

4. The analyst reads across each row and marks the task's 
highest loading (within sign) in the row. In the ex- 
ample listed above the highest loading within sign 
for Task 2 is on Factor 4 (-.0347), because the other 
loadings, though higher numerically, ahve a sign op- 
posite from the characteristic sign of the factor. 

5. When all the casks have been assigned to a factor the 
analyst reads the names of the tasks assigned to each 
factor. The work content of the factors should now 
emerge. 

a. A number of low-level tasks will seem inappropri- 
ate as assigned. These can be reassigned based 
on logic, as described earlier. 

b. A number of tasks may seem to be related to each 
other and inappropriate on any of the factors. 
These may reflect a relationship to a factor that 
did not emerge due to insufficient frequency of 
variables in the data base. These can be assigned 
to a separate non-factor grouping. 

c. A number of tasks may have very high loadings on 
two factors. These should be assigned in a man- 
ner that reflects the nature of the majority of 
tasks that load unambiguously on the factors. 

d. If there are any remaining ambiguities, final 
assignments can be made after the job level of 
tasks has been evaluated, as described in Chap- 
ter 3. 
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6. A list of tasks tentatively assigned to each factor 
should be prepared, listed in order by Task Code 
Number, with the factor loading included. In this 
listing and in subsequent reports we use the conven- 
tion of presenting the loadings with positive char- 
acteristic signs; we use the negative sign to repre- 
sent inverse loadings; thus, loadings with inverse 
signs are correctly interpreted as less than zero. 

7. The factors are given final name and number desig- 
nations which reflect the work specialty of the tasks 
that load on the factor. This may be an occupational 
reference and/or a functional reference. 

f 

Tt is possible for the analysis conducted up to this point 
to prove to be disappointing; i.e., the task factors may not "make 
sense" or may not be sufficiently differentiated to be useful. At this 
time it is in order to reconsider the PCVARIM factor solutions. In HSMS 
experience, the PCVARIM solutions sometimes make more sense after the 
consequences for task factors are seen. It then may be obvious that a 
larger or smaller number of variable factors can provide more interpret- 
able or useful task factors. After a new factor solution is chosen, the 
two-mode programs are rerun, and the analysis steps are repeated. 

It is the strength of the HSMS method that one is not "stuck" 
with any given solution. The one that most clearly accounts for the 
data and serves the ultimate purpose of the analysis is the best solu- 
tion; and it is never too late to reconsider which solution is most 
appropriate to the needs of the user after trying several solutions. 
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CHAPTER 3 



JOB STRUCTURES AND JOB LADDERS 

This chapter describes the use of the HSMS MATRIX program 
for assigninfc^, tasks to job levels.''" It is a manual for the structuring 
of jobs and '.he creation of job ladders based on the task factors and 
the job level analysis. The first section explains how MATRIX is used 
to assign an index of "difficulty" to tasks within factors based on 
skill and knowledge requirements. The second section describes the 
assignment of tasks to job levels. The third section discusses the 
design of job sequences or ladders and presents examples of HSMS rec- 
onmendations. The sequence of events is as follows: 

1, The tasks are arranged in rank order within factors 
by "point score" order of "difficulty," based on a 
count of all the skills and knowledges required for 
each task and the scale values at which each is re- 
quired (including variables not part of the factor 
analysis) • 

2. Tasks are assigned to job levels within each factor 
based on their "point scores" and "profiles." 



3. Once the tasks are assigned to job levels within ^ 
factors the results are examined and job ladder 
and lattice recommendations are made. 



DETERMINING RELATIVE TASK 
DIFFICULTY; POINT SCORES 

The hierarchy of tasks in a factor is the arrangement of 
tasks by order of difficulty so that tasks can be assigned to job levels < 
The MTRIX program is used for this purpose. 



See Appendix F for a description and listing of MATRIX. 
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I Factor loadings take account of the skill and knowledge 

variables that enter into the factor analysis; they do not reflect all 
the skill and knowledge requirements of tasks. For this purpose HSMS 
carries out a **point analysis** in which every skill or knowledge cate- 
gory needed for a task and the scale value at which each is required 
enters into a **point score** for each task. This is an index of educa- 
tional requirements, or **difficulty.** 

The HSMS MATRIX program allows the user to examine tasks 
separately for each factor- or in any grouping of interest. It presents 
the task data being examined in an array in which the tasks are arranged 
in columns from left to right in any order selected. The rows are all 
the skill and knowledge variables, listed from top to bottom in the or- 
der in which they appear in the tasks (as arranged from left to right) , 
including all the variables. The entries are the original scale values. 
Regardless of the order in which the tasks are arranged, by reading 
across a row one can see the tasks for which a given skill or knowledge 
category is required, at what scale values. 

Within a factor, some skills or knowledge categories are re- 
quired at the same scale value for all tasks, and some are required at 
more than one scale value, depending on the tasks. We assign points to 
the tasks based on what is found in the MATRIX array. The total is tlie 
task's point score. This is calculated for all the variables and tasks 
in a factor or in a selected set of tasks. 
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The specific analytic work with the MATRIX program to arrive 
at point scores is as follows: 

1, The user works with the tasks in one task factor at 
a time. If there is interest in comparing point 
scores across factors in order to decide on a factor 

• solution, or to decide on a factor assignment for 

given tasks, tasks assigned to more than one factor 
may be included. A given MATRIX "run" is the set 
of tasks that are to appear in a given MATRIX array. 
The resulting point scores are automatically com- 
parable across all the tasks in the "run," 

2, For purposes of point score analysis the user enters 
the tasks in a given "run" in numerical order by 
Task Code Number, This makes it easy to locate the 
tasks in the array. Any other order, such as by 
factor loading, is acceptable. 

3. The MATRIX print-out includes the following: 

a. A list of tasks in the array listed by Task Code 
Number and abbreviated name presented in the 
order entered. The tasks are given internal 
numbers, and the last number is the total num- 
ber of tasks in the array. 

b. The actual MATRIX array. The column headings 
are the Task Code Numbers in the order requested, 
arrayed from left to right (from lowest to high- 
est Task Code Number). The far right-hand col- 
umn identifies the rows, which are the skill or 
knowledge category codes, listed in the order 

in which they appear in the tasks as listed. 
The entries are the scale values with the deci- 
mals omiuted. (See example under 5, below.) 

c. A list of the skill and knowledge categories 
represented in the array, with the skills first, 
followed by the knowledge category code numbers 
in numerical order. The frequencies across the 
tasks in the array are alr,o given. 

4. When a given set of tasks is larger than the number 
that can be accommodated on the width of a computer 
sheet, the MATRIX row array is repeated, and the 
column headings are continued until all the tasks 
are listed. The analyst mur^t cut the continuation 
sheets and mount th»^m carefully to provide one con- 
s' 3 
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tinuoui* array. This is easily done by matching 
lines and headings and using glue or transparent 
tape . 

Below is an example of a MATRIX array as described 
above: 



EXAMPLE OF MATRIX ARRAY 
(First 7 Tasks in Task Code Order) 




E 
T 
C 



SKILL 
OR 
CATEGORY 



HUM INTR 
ORAL USE 
METHODS 
QUALITY 
IMPLICIT 
FINC ERR 
HUMN ERR 
12223000 
READ USE 
OBJ MANP 
WRIT USE 
11737300 
65620000 
11738000 



Point Score* 



* Entered during analysis. 

Figures were calculated for this sample. 



Code: 



Circle 

o 



1 point; Box 

Q 



2 points; Triangle = 3 points, 



The analyst decides on a symbol code or a color code 
for calculating the point scores. The example above 
uses a symbol code because this report is printed in 
black and white. A set of colored felt tip pens is 
preferable for coding. 

The analyst works with one row (variable) at a time. 
In the example above, the first row is HUM INTR (the 
Human Interaction Skill Scale). The analyst observes 
::he lowest scale value appearing in the row (30). 
Using the code for the lowest value (1 point = circle), 
the analyst marks each scale value with the code for 
one point. Then, using the code for the next higher 
value (50), the analyst marks each second highest 
scale value with the code for two points (box). If 
there were a third value, it would be marked with the 
code for the next higher value (a triangle) . QUALITY 
has the scale values of 15, 35, and 70, and all three 
codes . 
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8. When all the rows have been symbol-coded or circled 
with color-coded pens, the columns, which represent 
the tasks, are added, A circle counts as one point, 
a box as two points, and a triangle as three points, 
etc, A blank receives no points. The sums are en- 
tered at the bottom. These are the point scores for 
the tasks, 

9, The analyst records the task^s loading on the factor 
and its point score using the list of tasks printed 
as part of the output to MATRIX, This list can be 
duplicated, cut out, and arranged in point score 
order, ^ 



ASSIGNING TASKS TO JOB LEVELS 

The HSMS method assigns tasks to job levels by arranging 
the tasks in a factor so that tasks which require similar skills and 
knowledges at similar scale values are assigned to corresponding and 
appropriate job levels. The point scores are the basis for this as- 
signment of tasks to levels within factors. 

To assign tasks to job levels, the MATRIX program is run 
for each factor with the tasks arranged from left to right in order of 
their point scores . This ordering of the array produces a stepwise 
pattern because skills and knowledge categories that appear in low- 
level tasks continue to appear across the array, and, as higher-level 
tasks are added, indented new arrays appear for skills and knowledge 
categories not required for lower-level tasks, (See Figure 6, pre- 
sented later in this chapter.) 



2 



The order of point scores roughly approximates the order of factor 
loadings, but differences are inevitable. This is because variables 
enter into the point scores that were not among those selected for 
the factor analysis; and variables with little variance would not 
have high loadings on the variable factors. 
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We mark off on each row the first task at which a scale 
value changes to a higher value, and continue across the row, marking 
the first appearance of still higher scale values • We examine the ar- 
ray to note at which position (task) large numbers of skill and knowl- 
edge categories are required for the first time, and where the scale 
values first rise. These step-like demarcation points help us to as- 
sess when we have a change of job level. 

Often there is a dramatic point of change. In cases where 
the array shows no dramatic breaks we supplement our analysis by de- 
termining point score ranges for job levels and by examining the 
names of the tasks, using common sense to determine the difference 
between levels. 

It is usually possible to compare job levels across fac- 
tors, and/or to use the point score of a tasK on several factors as a 
benchmark when determining job levels. However, this is only true 
when the knowledge categories involved have been "broken out" to com- 
parable levels of detail. For example, in the case of the profes- 
sional-level (5) radiation physicist which emerged on a quality con- 
trol factor, it was impossible to compare point scores with other fac- 
tors. The knowledge categories required for the physicist are listed 
in the Knowledge System solely as broad-level subject categories, and 
are not "broken out" into fine-level categories; as a result the tasks 
have relatively low point scores. 

In assigning tasks to job levels, HSMS uses a specific con- 
vention in which all job levels are identified by number, general name, 
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and academic or functional requirements. These are presented in Figure 
5, below: 



Figure 5. TOE KSMS JOB LEVELS 





HSMS Job Levels 


Academic or Functional Description 


8. 


Specialized Advanced 
Professional 


MD's with specialization such as attend- 
ing radiologists, internists, surgeons; 
doctorates and post doctorates with exn^ 
perience* 


7. 


Advance*! 
Professional 


MD's in residency or doctoral candidates 
carrying out advanced functions • 


6. 


Specialized 
Professional 


Masters-level occupational study and ex- 
perience; supervision of profe»sionals. 


5. 


Professional 


At least four years of academic and/or 
specialized occupational education; may 
include masters level. 


4. 


Junior Professional; 
Supervisor 


Supervision and/or instruction of stu- 
dents and/or staff at levels 1, 2, and/ 
or 3; may be equivalent to baccalaure- 
ate degree-level specialty. 


3. 


Technologist 


Specialized technologist education; 
equivalent to associate degree level. 


2. 


Technician 


Specialized technical education; up to, 
including, and beyond high school, but 
less than associate degree level. 


1. 


Aide 


Entry level; up to. and including high 
school. 



Figure 6 is a hypothetical presentation of a MATRIX array in 
which the tasks are arranged in ascending point score order based on 
point score analysis. The assignment of tasks to levels is shown, and 
the stepwise pattern is apparent. 



Figure 6. MODEL OF "MATRIX" ARRAY OF SKILLS 

AND KNOWLEDGES BY TASK AND JOB LEVEL 



Skills 
and 

Knowledge 
Categories 


FACTOR I LADDER 


Level 1 


Level 2 


Level 3 


Task 

1 


Task 
2 


Task 
3 


Task 
4 


Task 
5 


Task 
6 


Task 
7 


Task 
8 


Task 
9 


Skill 1 


Co) 


(i.o) 


\2 . 0 1 


\2.0| 


nm 


A 




.^Toj 




* Skill 2 


(^:o^ 


[2.0 1 




1- 1. 


(IToA 








Skill 3 








|4.5l 


l4.5l 


I4TI 






A 


* Knowledge 1 
















(IT?) 




Knowledge 2 














A 






Skill 4 








(To) 












Knowledge 3 
















|6.0| 




* Knowledge 4 




















Knowledge 5 




















Knowledge 6 




















Knowledge 7 




















Knowledge 8 





















Point Score 



10 



10 



10 



19 



21 



28 



* Asterisk denotes variables that determined the factor. 

1, Tasks are listed from left to right in ascending 
order of point score. 

2, Skills and knowledge categories are listed from top 
to bottom in order of appearance in the task array. 

3, Tasks are assigned to levels based on increasing 
numbers of skills and knowledges required and their 
scale values. 

Not every skill or knowledge appears in all subse- 
quent, higher- level tasks. 

Scale values do not necessarily rise from level to 
level . 

Scale values may vary within a level. 



4. 
5. 
6, 



Code: circle = 1; box = 2; triangle = 3; double circle = 4; double box = 5, 
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Below is an example of how the task content of factors can 
be presented by job level, Task Code Number, name, point score, and 
factor loading: 



TASKS ASSIGNED TO QUALITY ASSURANCE, MATERIALS FACTOR 

BY JOB LEVEL, POINT SCORE AND FACTOR LOADING p. of 



, Task 
I Code 



Abbreviated Task Name and Job Level 



I 



Point Factor • 
Score Loading! 



Level 5 (Professional) 



557 



! 546 



etc. 




Level 2 (Technician) 



etc. 



192 



Level 1 (Aide) 

Inspecting, checking, preparing xeroradiography 
equipment for use.* 



56 



Collecting and presenting technical information 
about and/or recommending new diagnostic x-ray 
equipment • 

Designing, maintaining, evaluating radiation pro- 55 
tection and monitoring programs in diagnostic 
radiology • 



14 



,79 



• 72 



28 



Calibrating diagnostic x^-ray test, survey, or mea- 28 

suring instruments. 
Checking automatic exposure texmination of diagnos- 27 .07 
tic radiography equipment. 



.07 



1 



The following explanation is given to the reader in the form 
of footnotes or notes: 



ERLC 



Note: Tasks are arranged in descending order within levels by 
point score (difficulty as reflected in the number of 
skill and knowledge categories required and the scale 
values at which the categories are required). Factor 
loadings run from high, positive values, through zero, 
to negative values (for lower-level tasks). 

* Each uask was assigned to the factor on which it has its 
highest loading. Task names marked with an asterisk (*) 
were assigned on the basis of logic. 
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DESIGNING JOB LADDERS AND LATTICES 

The tasks assigned to any given level within a factor are 
likely to be representative of the central tasks of a job. Naturally, 
any job will also include certain peripheral tasks not on the factor 
which reflect the administrative or institutional idiosyncrasies, paper 
work, conferences, etc», usually associated with any job. In some 
cases a real job would combine the tasks in more than one factor, such 
as when an institution is not large enough to differentiate jobs. How- 
ever, for the purposes of job or educational ladder design, the tasks 
at a given level within a factor suggest the most rational assignment 
of major duties to a job, since they require the maximum application 
of a minimum but adequate educational investment. 

The factor results do not necessarily guarantee that job 
ladders can be designed within task factors. After tasks have been 
assigned to factors and levels it is possible to find that only one 
level is represented in a given factor. However, some factors that 
appear only at higher job levels can be logical higher-level steps for 
tasks at lower job levels in other factors. This happens when the 
higher-level factor combines the skill and kno\%ledge requirements of 
one or more other factors. This becomes the basis for constructing 
job lattices . 

The HSMS job ladder and lattice recommendations are arrange- 
ments of jobs in promotional steps derived from the task factors. The 
jobs in a ladder or lattice require interrelated skill and knowledge 
categories based on given task factors. Job lattices allow for lirk- 
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ages ecross factors both hori-/,ontally and diagonally. This provides 
crossover options and a choice of promotional pathways based on the 
principle that the skills and knowledges required at a given level 
for a factor may serve as a basis for more than one specialty. A 
given specialty may build on more than one kind of prior preparation; 
the entry to specific professional jobs could thus be reached through 
more than one factor. Conversely, a given job level in a factor can 
be a step towards more than one specialty. 

The analyst makes recommendations on job structures, job 
ladders, and job lattices based on the nature of the task factors 
which emerge and on the assignment of tasks to job levels. Once these 
are laid out and the skill and knowledge content is examined, logical 
recommendations usually emerge. 

Past HSMS analyses are presented in Figures 7, 8, and 9* 
Figure 7 is an example of the factor structure that emerged in diag- 
nostic radiology; Figure 8 shows the job ladder-lattice recommenda- 
tions which resulted from the analysis. Figure 9 shows a similar set 
of recommendations including diagnostic ultrasound and radiation 
therapy. 

These examples are offered as indications of HSMS recom- 
mendations, since it is impossible to provide more detailed instruc- 
tions on how to go from the factor and point score data to the actual 
suggestions. 
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Flyure 7. SUMfiARY OF FACTOR STRUCTURE OF TASKS BY JOB LEVEL; DIAGNOSTIC RADIOLOGY 



Factors: 



Job Levels 



Non-Neuro. 
Radiology 



II 



Neuro- 
Radlology 



V 



Obs-Gyn 
Radiology 



8. 



Special- 
ized Ad- 
vanced 
Profes- 
sional 



7. - 
6. - 

5. Profes- 
sional 



Radiologists 
(non-neuro, 
radiology 
specialties) 



Radiologist 
(neuro- 
radiology 
specialty) 



Radiologist 
(obs-gyn 
radiology 
specialty) 



4. Jr. Profes- 
sional; Su- 
pervisor 



3. Technol- 
ogist 



2. Techni- 
cian 



1. Aide 



*Tasks in box with asterisk (*) 
not covered by data. 
Source; Research Report No. 8. 



IV 



Patient, 
Emergency 
Care 



(consultation; reading, interpreting; contra© L 
studies; research; residents* training) 



Chief Tech. 
of Pt. Care 
(teach; give 
emerg.care) 

^T^^ 



injct;lst aid 



Pt.Care Tech- 
nicn (cath. ; 
bandfr; prep. 
specimens) 



III 



Radiologic 
Technology 



Chief Rad. 
Tech. 

(teach ;eval, 
techJs work) 



Rad. Tech. 
(plain, con- 
trast pt. 
examinatns ; 



operatg.rm*) 



Pt.Care Aide 
(asst.;vital 
signs; ECG; 
pers.atten.) 



VI 



Quality 
Assurance 



NON-FACTOR A 



Adminis-- 
tration 



Radiation 
Physicist 
(design, run 
qual.assur . 
programs) 



jDept. Ad- j 
jministrator 
((manage; ' 
|purchase ; ' 
ijbudget)* I 

T ■ 



Supervisor 
(eval.sub.'s 
work; run 
meetings) 
T 



Admin. Tech. 
(inventory; 
scheduling; 
orientation) 



Qual. Assur, 
Technician 
(test x-ray 
equipment) 



Qual. Assur 
Aide(process 
films ;prep 
equipment) 



Figure 8. SUMMARY OF JOB STRUCTURE AND CAREER LADDER RECOMMENDATIONS 



Factors : 
(task group- 
ings) 

Job Levels 

5. Profes- 
sional 



Jr. Profes- 
sional; Su- 
pervisor 



3. Technol- 
ogist 



2. Techni- 
cian 



3 . Aide 



IV 



Patient and 
Emergency 
Care 



CHIEF TECH. OF 
PT. CARE (teach, 
evaluate) or 
EMERGENCY CARE 
SPECIALTY 



PATIENT CARE 
TECHNICIAN (in- 
ject^lst aid; 
cath. ; bandg. ; 
prep > s pec imnsQ 



PATIENT CARE 
AIDE (asst.; 
vital signs; 
ECGj personal 
attention) 



III 



Radiologic 
Technology 



CHIEF RAD. 
TECHNOLOGIST 
(teach , evalu- 
ate subordns.' 
work) 



RADIOLOGIC 
TECHNOLOGIST 
(plain, con- 
trast pt. ex- 
aminations) 

TS. 



VI 



Radiologic 

Quality 
Assurance 



RADIATION 
PHYSICIST 
(design, ruxi 
qual.assur. 
programs) 



NON-FACTOR A 



Adminis- 
tration 



DEPARTMENT 
.ADMINISTRATOR 



(manage; plan; i 
jpurchase ;bud- i 



SUPERVISOR 
(evaluate sub- 
ordinates' workj 

run meetings) 
^ — 



[ADMINISTRATIVE 
TECHNOLOGIST 
(inventories ; 
scheduling; 
orientation) 



QUALITY ASSUR. 

TECHNICIAN 

(test x-ray e- 

quipmt^monitor 

processors) 
7K 



QUALITY ASSUR- 
ANCE AIDE (pro- 
cess films; 
prepare, clean 
equipment) 



* Tasks in box with asterisk (*) not covered by data. 
Source; Research Report No. 8. 
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Figure 9- RADIOLOGIC TECHNOLOGY, ULTRASOUND AND RADIATION THERAPY: j 
RECOMMENDED JOB STRUCTURES AND PROGRESSIONS BY TASK FACTOR AND JOB LEVEL 



Task 
Factors: 

Job Levels 

5. Profes- 
sional 



Jr* Prof- 
essional 



Technol- 
ogist 



2. Techni- 
cian 



1- Aide 



Diagnostic 
Ultrasound 
Technology 



Patient and 
Emergency 
Care 



Diagnostic 
Radiologic 
Technology 



t 



SUPERVISOR- 
EDUCATOR 



» ! CLINICAL NURSH 
J L S P E C lA L T Yi 



ULTRASOUND 
TECH, : pt, ex- 
aminations in 
all modes 



SUPERVISOR- 
EDUCATOR 



RADIOLOGIC 
TECH, : pt. ex- 
aminations V, 
w/o contrast 

3^ 



PATIENT CARE TECHNICIAN: Injec- 
tions, medications, teach self 
care, 1st aid, insertn. of probes 
catheters: all 3 departments 



Radiologic 

Quality 
Assurance 



RADIATION PHY- 
SICIST: design, 
run qual, ass- 
urance progrms> 



QUALITY ASSURANCE TECHNICIAN 
test diagnostic equipment > moni- 
tor film processors , prepare 
transport materials 



PATIENT CARE AIDE: Assist pt., 
vital signs , reassure, assist 
with procedures, ECG, personal 
attention: all 3 departments 



Radiation 

Therapy 
Technology 



SUPERVISOR- 



L ^EDUCATOR _ j 



Admin- 
istra- 
tion 

*1)epSit-"^ 

* MENT I 
I ADMINIS-i 
l_ TRATOR J 

"1 ^SIJPER-'^ 
L V I S^O R ^ 



RAD, THERAPY 
TECH, : set up, 
administer, co- 
ord, treatments 



RAD, THERAPY 
TECHN, : mold 
room work, as- 
sisting 



I 



QUALITY ASSURANCE,MATERIALS 
AIDE: checking, preparing mate- 
rials, inventories, film pro- 
cessing: all 3 departments 



[ADMINIS- 
TRATIVE 
TECH, : 
schedules 



Broken boxes repres'^nt jobs whose tasks are not fully covered by data. 
Source: Research Report No, 10. 



Once the job structure and job ladder tecommendations are 
made, the user is ready to prepare the data for use in curriculum out- 
lines, curriculum objectives, and educational ladders to parallel the 
job ladders. This is discussed in Volume 4 of this report. Chapter 
4, which follows, describes the use of the data for institutional 
evaluation and planning. 
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CHAPTER 4 

USING TASK DATA TO MAKE RATIONAL USE OF MANPOWER 

Prior chapters described how statistical analysis of HSMS- 
type task data ^can be used to develop job structure and job ladder 
recommendations. This chapter shows the director or administrator 
of a local institution how to adapt the recommendations, based on na- 
tionwide practices, to the needs of an individual institution. The 
chapter is directed to the hospital or department administrator who 
wishes to implement job structure and career ladder recommendations 
resulting from task analysis, or who wishes to use task data for per- 
formance evaluation. 

An institution cannot be expected to provide upward mobility 
to its employees unless it has practical reasons for doing so; it must 
make economic sense. The first section of this chapter argues that 
there are bonef its to . e derived from using HSMS-type data and recom- 
mendations. It discusses the economic rationales for job structuring, 
restructuring, and the development of job ladders using HSMS methods. 

The second section describes how the administrator can use 
the data generated ^y HSMS methods to rationally structure or res true- 



Much jf the material presented in this chapter appeared as Chapter 3 
in Research Report No. 8. (Fleanor Gilpatrick, Using Task Data in 
Diagnostic Radiology , Volume 1, "Job Ladders: Assigning Tasks to 
Jobs," New York: Health Servic-s Mobility Study, 1977.) 
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tvre jobs. It shows how to examine the allocation of tasks to levels, 
and describes the creation of new jcbs. The third section discusses the 
development of a career ladder program, cost saving strategies, and 
trainee selection. The fourth section deals with the use of HSMS-type 
data to evaluate institutional performance. It provides a mini-manual 
for using HSMS-type task data to create performance evaluation instru- 
ments. 

RATIONALES FOR JOB RESTRUCTURING AND CAREER LADDERS 

The HSMS method makes it possible to analyze tasks in terms 
of their skill and knowledge requirements and their relationship to 
other tasks and job levels. It is therefore possible to assign tasks 
to job titles and to make optimum use of more highly trained and more 
expensive employees, and to make sure that the work is being carried a 
out by staff who are properly trained to provide quality input. 

The assignment of tasks to job titles is job str uc turing or 
restructuring . The arrangement of jobs into a promotional sequence 
from one level to another is job ladder construction . It is not always 
necessary to do job restructuring in order to design and implement job 
ladders; it is also possible to derive advantages from job structuring 
or restructuring without having to arrange jobs into a promotional lad- 
der. This is discussed below. 

The costs to consider in structuring or restructuring jobs 
are thos of salary and education. Direct education or tuition costs can 
be borne by students, employers, or society; education costs, however, 
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are also reflected in salary levels. The education time needed to pre- 
pare for jobs is highly correlated with salary levels. Reference to 
high-level staff or jobs implies high salaries, high skill and knowl- 
edge requirements, and long, expensive periods of educational prepara- 
tion. Conversely, reference to low-level jobs implies low levels of 
educational preparation and low salary levels. 

Job Structuring and Restructuring 

Job structuring and restructuring can provide cost advantages 
if tasks are assigned to jobs so that the skill and knowledge levels re- 
quired for tasks are compatible with the educational and salary levels 
of the jobs to which they are assigned. Allocation of low-level tasks 
to high-level jobs is wasteful of salary and education costs. If there 
are shortages of high-level personnel, the waste is felt as decreased 
services or other outputs. 

It also makes economic sense to assign tasks to jobs so that 
the skill and knowledge requirements for all the tasks in a job are sim- 
ilar. Assigning tasks requiring different, non-overlapping skill and 
knowledge requirements to a single job increases the amount of educa- 
tional preparation needed to do the job, even if all its tasks are at 
the same level. This prolongs the educational preparation time needed 
and probably inflates salary levels. 

Job structuring and restructuring may be done to make jobs 
at lower levels less boring for workers in order to improve morale and 
thereby improve performance and/or reduce turnover costs. Such *'job" 
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enlargement" can be done economically by increasing the variety of task 
activities in a job while still assigning tasks which require the same 
basic investment in skill and knowledge training. 

Job structuring is needed when the institution is to provide 
a new service or function, or is to utilize a new technology. It is 
then important to identify the tasks involved, to decide the job levels 
to which the tasks should be assigned, and to decide the existing job 
titles to which the tasks can be assigned to minimize the need for new 
educational preparation. It is important to clarify whether there is 
justification for creating entirely new jobs and/or whether the develop- 
menu of a job ladder is appropriate. The, issues are discussed later 
in this chapter. 

Job Ladder? 

Job or career ladders provide upward mobility for the in- 
house labor force of an institution. Promotional lines provide for a 
supply of new entrants inco jobs as older incumbents retire, are dis- 
missed, or as more staff are needed in a job title to handle increased 
volume. 

The most powerful economic reason to have a career mobility 
program is to fill chronic vacancies at middle and upper job levels. 
In a field such as healt' services, most promotional lines require ad- 
ditional education as an individual goes from one job level to another. 
An economically desirable career raobility ogram would provide job 
ladder sequences that minimize the additional education needed between 
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levels. If a job ladder starts from an entry level job with few vacan- 
cies, and progresses from one job level to another within interrelated 
task groupings to the level where shortages exist, the amount of educa- 
tional investment required between each level is minimized; employees 
need to be trained only for the educational gap between one job level 
and the next. 

There are other economic arguments in favor of job ladders. 
By selecting in-house incumbents in appropriate jobs to move up a job 
ladder, the institution can cut the costs that are incurred in orient- 
ing new employees. A program of upward mobility can also become an in- 
centive for efficient performance if selection for upgrading is partly 
dependent on the quality of performance in the current job. Since 
trainees currently successful at one job level are likely to be suc- 
cessful at the next level (because of similar job content), the failure 
rate may be reduced. A career ladder program may also reduce the costs 
of turnover to the degree that high turnover reflects di.3couragement 
with "dead-end" jobs. 

Actual salary costs may be lower with the use of upgrading 
programs than if staff are hired from the outside. The staff selected' 
for upgrading will be at the top of their salary range when selected, 
but will enter the new jobs at the bottom of the salary range. They 
will be replaced in their former jobs by staff who are themselves newly 
upgraded and who will be entering at the bottom of the salary range for 
their new jobs. 
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Competition among institutions to attiact and hire outside 
individuals whose training is in short supply creates an inflationary 
pressure on salary levels. An in-house career mobility program adds to 
the supply of scarce labor and reduces inflationary pressures. Strate- 
gies for job ladder construction are discussed later in this chapter. 

When shortage jobs are at a high level, with no related jobs 

at intermediary levels, joj structuring or restructuring may be needed 

to provide job ladders . If the educational distance from an entry job 

i to a shortage job is a matter of several years, one cannot talk about a 

viable job ladder. For example, a one-step rise from the aide to the 

professional level would be unrealistic. But a ladder from the aide to 

the technician level, and from there to the technologist level can ulti- 

2 

mately lead to the professional level in reasonable stages. 

To create a new job as an intermediary level on a ladder or 
to provide a new service or function is a form of specialization of 
labor that can be cost saving within limits. As different components 
of work are separated out and assigned to different jobs, the work can 
be done more efficiently and more economically. Lower-^ievel t£»^ks can 
be grouped into jobs at lower salaries. The limit to this approach is 
that the institution must be large enough to provide fall-time work in 
each of the subdivided specialties. Short of this, workers would not be 
efficiently utilized. This question is discussed later in this chapter. 



It is important to note that a job ladder progression refers to the re- 
lationship among job title s . A given individu al may not move up on all 
the rungs of a ladder. At any point in time incumbents at one leval on 
a ladder are the population from which those who will be trained to go 
to the next level on the ladder are selected. 
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USING TASK DATA TO STRUCTURE JOBS 

Assuming that the administrator of a department is interested 
in the rational structuring of jobs in the department, HSMS-type methods 
can provide the raw materials. The HSMS method produces task identifi- 
cations and descriptions, groupings of tasks, assignments of tasks to job 
levels, and results in job structure and career ladder suggestions. The 
administrator can adapt data, analyses, and recommendations that reflect 
nationwide practice for use in his or her own department. 

Analysis of Job Structures 

Data Preparation 

In adapting the recommendations the administrator first de- 
cides on the job titles to be examined; the second step is to identify 
the tasks being carried out in those titles; the third step is to ana- 
lyze the pattern of distribution of the tasks in terms of task overlaps 
across jobs, the levels of tasks in jobs, and the groupings of tasks 
in jobs. 

The :in]inistrator starts by selecting the job titles to be 
examined. These are placed on a reference list. The list should in- 
clude all the in-house titles of interest along with the salary or sal- 
ary range for 3ach. Next, a HSMS job level should be assigned to each 
job on the list. Figure 5 in Chapter 3 presents the HSMS job levels. 
A way to check the r.ppropriateness of job level designations is to note 
whether the rank order of the job titles by salary level is the same as 
the rank order of job titles by HLMS job level designation. 




The next step is to determine which individual(s) are famil- 
iar with all the work being done by all the incumbents in the job titles 
on the list. This may be the administrator, or different supervisors 
may be familiar with different titles. These individuals are "respond- 
ents" or resource persons who can be asked to provide the basic informa- 
tion on the current allocation ol tasks to titles. 

The next major step in the analysis is to determine which of 

the tasks covered in the nationwide analysis are being carried out in 

job titles in the department or in titles related to the department. It 

may be best to get an overall sense of where the activities are bein^j 

3 

done before getting detailed information for each title. The finnl list 
should include only tasks being done at the institution. 

The next step is to find out in which job title or titles 
each task is done. For each in-house job the entire list is considered 
to ensure that all the tasks for a given title are covered. This means 
that a copy of the task inventory of all tasks should be prepared for 
each interview with each respondent. If it is decided that it is im- 
portant to know which task^ are carried out by individual employees in 
a title, separate lists should be prepared for each; and each should 
be given the appropriate job level designation by title. 

Each respondent is asked to indicate which tasks in the in- 
ventory are carried out in a given title. If a true representation of 

The HSMS extended task names provide good content references: they are 
sufficiently detailed to avoid confusion about what work activities are 
being referred to, 

4-8 



ERIC 



72 



work assignments is of interest, an effort should be made to include out- 
of-title work, because this may be of major economic significance. Since 
this it an in-house analysis, security is not an issue. 

At this stage there is a task list for each title and/or in- 
dividual being studied. Each contains all the tasks done in that job. 
(The Task Code Numbers and abbreviated task names are useful references 
at this point.) Next to the name of each task, two additional pieces 
of information are needed. The first is the HSMS job level designation; 
the second is the name or number of the factor to which the task was as- 
signed in the prior analysis. 

A third piece of information may be of interest to the ad- 
ministrator. That is the frequency with which the task is carried out 
in a given job. This information will be helpful if there is interest 
in the relative importance of a task in the structure of a job. The 
basic information is obtainable from the respondents. (The use of the 
HSMS Task Frequency Scale (Scale l) is discussed in Volume 2 of this 
report. ) 

If there is an interest in ove rall assessment of t^s manpower 
utili/.ation pattern in the department , the next step is to create an ar- 
ray thac contains the information of interest. This is done by arrang- 
ing the job titles (and the names of individuals within titles) in col- 
umns , from left to right in descending order by HSMS job level and/or 
salary level. Within job levels the titles shduld be arranged by task 
factor. The factor for a job is determined by the most prominent fac- 



tor showing on the task list collected for the title (or individual). 
The rows in the array are to be all the tasks found in the department, 
arranged from top to bottom in descending order by HSMS job level, and 
within job levels by the same factor order used for the columns. The 
entries in the array are x's. Working with each list separately, one 
fills in a column at a time, placing an "x'* in the appropriate column 
if a given task is found on the given job's list. Figure 10 is a hypo- 
thetical example of such an array \n which HSMS data were used for the 
task numbers, levels, factors, and tides; the example assumes twelve 
incumbents, listed by number. 

The array provides an overall view of the extent and loca- 
tion of task overlap and the appropriateness of current allocations of 
tasks to job titles by levels and factors. One examines the overlap of 
tasks acro:^s job titles or incumbents by reading across the array; one 
examines the mix of tasks in jobs by level and factor by reading down 
the columns. An ideal utilization pattern would be roughly in the 
shape of a diagonal, falling from left to right (as wide as the adja- 
cent columns in a given factor; as high as the adjacent rows in a given 
factor within a level), P'igure 10 shows this pattern, with the exception 
of Tasks 490, 74, and 275. 

Task Overlap 

Task overlap occurs when a task is carried out in more than 
one job title (or across mere than one incumbent of a title if there 

To save space, abbreviations and Task Code Numbers can bo used de- 
signate rolumns and rows, since the entries will be x*s. 
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Job Levels^: 1 


J 


5 


4 


3 


2 


1 


Factors^: I 


II 


VI 


III 


IV 


III 


A 


IV 


VI 


IV 


VI 


Job Non- 
Titles^: neur, 
Rad. 


Neu- 
ro. 
Rad. 


Phy- 
si- 
cist 


Rad. 
Tech, 
Supr. 


Pt. 

Care 
Supr. 


Rad. 
Tech. 


Ad- 
min, 

I'ech. 


Pt. 

Care 
Tech. 


Qual 
Ass. 

Aide 


Pt. 

Care 
Aide 


Qual. 
Ass . 

Aide 


Incumbents: 1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


11 


12 


Le- 
vel 


Fac- 
tor 


Task 
Code 


























8 


I 

TT 

J. L 


441 
329 
448 

397 
430 


X 
X 

. X 


X 

y 

X 






















5 


VI 


528 
546 
542 






X 
X 
X 








1 












4 


III 

TV 

J. V 


82 

7 

305 








X 
X 


X 
X 
















3 


III 

A 


526 
362 
363 
496 
131 
272 












X 
X 
X 
X 


X 
X 








1 

j 






2 


IV 
VI 


299 
33 
143 
308 
243 
535 
548 
549 
















X 
X 
X 
X 
X 


X 
X 
X 




1 




1 


IV 
VI 


290 
190 
193 
490 

74 
147 
275 

t9 
552 

79 


X 
X 

1 










X 

1 








X 
X 
X 
X 
X 


X 
X 

ix 


X 
X 
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educator, supervisor; level 3: technologist; level 2: technician; level 1: 
aide. 

Factor I: Non-neurologic Radiology; Factor II: Neuroradiology; Factor III: 
Radiologic Technology; Factor IV: Patient Care; Factor VI: Quality Assur- 
ance; Non-factor A: Administration. 
Fill in the in-house titles. 
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are different jobs within a title). Not all overlap is undesirable or 
avoidable. There are always overlap tasks to be done which are the 
mortar that holds the central tasks in a job together. There can be 
duplications that reflect the different locations or shifts in which 
the work is carried out. However, when there is duplication of the 
central work in a given department, this bears close examination; thus, 
the overlap data in the array should be given careful analysis. 

The most important type of overlap to look for is where the 
same task is found in jobs that are at different I'Bvels . The allocation 
of low-level tasks to high-level titles is wasteful. If high-level task.^> 
are allocated to low-level titles this may imply inadequate performance 
or wasteful training. Given acceptable performance of a task in the 
titles where it is currently overlapped, there is a prima facie economic 
argument for downward assignment of an overlap task to the lowest level 
in which it is currently found. In Figure 10, Tasks 490, 74, and 275 
are such tasks . 

Sometimes the overlap reflects the case where supervisors fill 
in for absent staff. This may be a waste of expensive supervisory time. 
One solution might be to develop a "flying squad'' for lower-level jobs. 
Such staff would be trained for several jobs at the aide level and would 
be on call to fill in for absentees. The squadfs) x^ould provide a source 
of experienced manpower to cover staff absences at the aide level. By 
virtue of this experience, employees at the aide level could later ^ake 
informed choices about the specialty in which they would like to rise. 
Management would be in a position to take account of especially gifted 
employees and encourage them. Rotation would permit job enrichment. 
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Sometimes overlap of tasks across job levels reflects the 
refusal of professional staff to delegate work. Some professionals v^e- 
fer to carry out lower-level tasks when they are not comfortable about 
the quality of the performance of lower-level staff. Discovery of this 
kind of overlap actually pinpoints job performance and training inade- 
quacies. The solution is to provide remedial training so that higher- 
level staff can rely on the quality of work assigned to lower-level 
staff. 

The in-house analysis of the overlap data should result in 
the separation of necessary from unnecessary task overlaps, a design 
for the rational restructuring of jobs, and a list of steps needed to 
remedy the problems uncovered. 

Job Structure By Task Level and Factor 

At this stage it is possible to analyze whether the tasks in 
each job are at appropriate job levels. Allocation of tasks to jobs at 
comparable job levels is the economic objective. Since most jobs can- 
not be held together without one or two tasks that are essentially 
simple and/or admi.tistrative, percentage distributions and task fre- 
quency data are used to examine whet er current allocations are sound. 

The allocation of lower-level tasks to higher-level jobs sug- 
gests waste. It is also important to consider the presence of higher- 
level tasks in lower-level jobs. In a case where a task is rated at a 
level higher than the job in which it is found, the task may be beyond 
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the reach of the incumbent's experience and training, and performance 
may be unsatisfactory. Alternatively, the staff in this job may be re- 
ceiving training for the one task at levels that are beyond the needs 
for all the other tasks of the job, and this would be wasteful of 
training • 

The analysis of the composition of jobs by task factor is 
similar to the analysis of the task levels. The array and the lists 
provide insights about the breadth of training required for the jobs. 
A job made up of tasks that cross several factors may require training 
in a larger number of subject areas than is economically warranted. 
For example, if the same staff member were administering medication and 
testing x-ray equipment, an investment in training in pharmacology sub- 
jects anc* in technological subjects would be required. With no trans- 
ferability from one to the other, and no likelihood that this combina- 
tion would be foi.nd in other lateral or higher jobs, we have a wasteful 
job structure. 

Creation of New Jobs 

A new job may need to ^ e created as a result of the analysis 
of task allocations described above, or to provide an intermediary job 
between high and low-level jobs, or to provide for a new function, or to 
utilize a newly available technology. The key to structuring a new job 
is to know all the tasks to be covered, their job levels, and their fac- 
tor designations. Frequency data reflecting the expected work loads to 
be assigned are also helpful. 
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The decision to institute a quality assurance program in di- 
agnostic radiology provides a case to illustrate the type of analysis 
that might be considered in the creation of new jobs, given the princi- 
ples already described. HSMS found that the tasks in this case fall into 
two job levels, those of the technician and those of the professional • 

Quality assurance tasks are not found in every hospital, and 
not all quality assurance tasks are present in any one hospital. Those 
that are present can be found in a variety of job titles and job levels. 
Now that there is interest In instituting quality assurance programs, it 
would be sensible to decide on appropriate job structures to contain the 
technician and professional-level tasks. It would be cost-saving to 
remove technician-level tasks from their current locations in technolo- 
gist and professional-level jobs and co create a quality assurance tech- 
nician job. The remaining professional-level tasks would be assigned 
to the professional radiation physicist. However, this solution makes 
sense only if the institution is sufficiently large to benefit from a 
newly created quality assurance technician job. 

In the absence of sufficient volume, the HSMS designation of 
level and factor for tasks can be used to decide the best allocation of 
the technician-level tasks among existihg jobs. The decision can vary 
for different institutions depending on frequency data for current tasks; 
however, the fr owing reasoning would apply. 

The technician tasks can be taught to the aide and added to 
the aiders current duties. The new costs would be for training and a 

salary increase, because the aide job would now include technician-level 
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tasks. It is also possible to teach the tasks to the :echnologist and 
add them to the technologist's current duties. The nev costs would be 
those foi *-aining; there would also be hidden costs due to less than 
optimum use of the technologist's time when carrying out technician-level 
tasks. With the use of data on frequency and current work loads and 
flows, a sound economic decision cwx be reached. 

What is inescapably apparent is that there is little justi- 
fication for assigning the technician tasks to the professional, who is 
an expensive employee. If the reason for a job structure in which the 
professional is doing the technician-level tasks is that this is needed 
to provide full-time work for the professional physicist, the true func- 
tion of the physicist may not be properly understood, and appropriate 
professional tasks may be missing- In addition, it may be sensible for 
a small institution not to hire the professional as a full-time employee, 
but to retain a consultant who will set up and run the quality assurance 
program as needed. 

CAREER LADDERS AND COST SAVING STRATEGIES 

An institution may decide to develop a program for upgrading 
staff in job ladder progressions because it is convinced that this ap- 
proach is most efficient in the long run; it may have decided that this 
is the way to expand the services it provides, whether in sheer quantity 
when demand increases, or in the provision of new services or functions. 
The institution may have decided that this is the way to fill chronic 
vacancies. Its commitment to upward mobility may have been brought 

about through collective bargaining, with a portion of the wage package 
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set aside for the upgrading-training of staff • In any of these circum- 
stances there are basic decisions to be made that can affect costs and 
the success or failure of the program. This section brings together va- 
rious insights gained by HSMS about the cost aspects of career mobility 
programs • 

Overview 

Unlike the situation where students gain their occupational 
preparation before they enter the labor force, an upward mobility pro- 
gram is concerned with the needs of working students, with employers, 
and with educators. The students are employed adults who very likely 
are the main source of sui^port of themselves and their families. Em- 
ployers may wish to provide *"heir staff with occupational preparation, 
but also need to maintain the quantity and quality of their productive 
output 

In health services occupations the jobs require instruction 
in formal disciplines. The subject matter must be imparted by teachers 
and learnv^d in ^he classroom and in supervised clinical practice. Un- 
like many factory or civil service staff, the health worker cannot **pick 
up" what is needed in the higher-level job by simply observing other 
workers during the course of a work day in his current job. In addition, 
entry to health services jobs is often circumscribed by requirements 
such as licensure, certification, graduation from AMA-approved or other- 
wise accredited programs, and/or academic degrees. In most cases licen- 
sure and certification require graduation from accredited prog'-ams as 
well as passing examinations, 
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Given these considerations, an in-house upward mobility pro- 
gram must involve four basic types of costs. These are education costs, 
released-time costs, relief worker costs, and trainee failure costs; 
there are alternative ways of dealing with each. 

Education costs cover classroom instruction and clinical 
practice, Thec^e would be faced by anyone entering study for an occupa- 
tion. The options and choices about which this chapter comments are 
as follows : 



1. There can be an in-house (hospital-based) program in 
which the institution runs the program; or there can 
be an academic program in which a student accumulates 
academic credits towards a degree at the associate, 
baccalaureate or masters level. 

2. The program can be designed as an educational ladder 
with course work sequenced so that the whole program 
leads to the top of the ladder and shorter segments 
lead to lower-level jobs, so that students can exit 
and reenter the program at job-related intervals; or 
there can be discrete programs designed for each job. 

3. Time Si:hedules for instruction can be geared to full- 
time students and regular academic semesters; or they 
can be gea- eC to the time requirements of employed 
students. 



Released-time costs are payments to trainees while they are 
studying to permit them to maintain incomes. The options include find- 
ing outside i?ssistance to pay employees, counting these costs as fringe 
benefits along with health insurance and passing them along as produc- 
tion COST'S to third-party payers, and/or having employees and/or educa- 
tional institutions share in the costs. 
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Relief worker costs cover the salaries of employees who will 
provide the relief work while trainees are studying. Among the options 
are hiring temporary employees to provide the relief work for Individual 
trainees , or using a staged approach in which the workers who will re- 
place the trainees in their former jobs when the latter are upgraded are 
the ones to provide the relief work. A strategy for this is discussed 
below. 

Trainee failure costs are incurred when trainees fail in 
their upgrading-training programs and are not able to fill the upper- 
level jobs. The selection cri**eria for trainees can affect success cr 
failure. A set of alternatives is discussed below. 

Education Costs 

Sequential educational programs based on job ladders save 
education costs by eliminating redundant education and providing rein- 
forcemeit and transferability of training. In addition, it is more 
economical in both the short and long run for health care institutions 
to give up the production of educational programs at technician and 
technologist levels. If they combine into consortia on a city-wide or 
system--wide basis they can p^urchase educational programs from academic 
institutions which can offer accredited programs and academic credits 
usable to\%ard college degrees. 

If large numbers of students are Involved, the educational 
institutions could be persuaded to offer programs that are properly 
timed and sequenced to service the career ladder programs adopted by 
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employeis, Tlie movement to work/study, continuing education, and work- 
oriented timing for course hours has been growing in colleges and uni- 
versities since the late 1960 's. Consortia can be created of employ- 
ers in a system, tuch as in a municipal or voluntary system, or in a geo- 
graphic area. Their function would be to adopt mutually acceptable job 
ladders and to purchase educational programs for a consortium's pool of 
trainees. 

The alternative is having health care delivery institutions 
provide internal training for their manpower needs. The training pro- 
duced is often so specific to the needs of the institution that the 
trainee finds it of little ust for upward mobility or even for lateral 
movement in the job market. This is particularly true in the so-called 
"new career" titles. Since the institutions themselves are not per- 
mittsd t:> provide academic credit?, thk. training is of no help in the 
attainment of the degrees which are a part of the credential system and 
are needed for higher- level jobs. 

The current time requirements for accredited occupational 
programs is a good argument for using the necessary time for students to 
accumulate degree credits as well as occupational certification. 

Aide-level training should include remedi£»tion and be used 

to ready workers for later educational rograms. It might be best to 

provide this in conjunction with programs leading co high school e ^uiv- 

alency diplomas or college-level credits. Everyone at the aide level 

should have the chance to receive high school equivalency training, 

especially credits in the high school subjects required for entry to 
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associate or baccalaureate-degree programs • Aides should be able to 
receive credit for their work experience vben this is appropriate • 

Given the numbei of trainees for upgrading programs that em- 
ployer groups or consortia can offer, educational institutions could 
reduce per capita costs through the use of facilities and faculties in 
courses offered in the evening, on weekends, during vacations, and at 
other non-peak times. In health services, member hospitals would be 
natural affiliates for the clinical^ training. 

A system-wide consortium apprca»':h could combine employers, 
educational institutions, and the relevart employee trade i^nions and 
professioral associations. Consortia could make maximum use of federal 
state, local, and foundation finding ^or its programs • It is a full- 
time job to locate the funds, write the proposals, and put the packages 
together. This can be done efficiently on a large, city-wide or system 
wide basis . 

Heleased-Time ai d Relief Costs 

It is desirable to keep Gtudents employed and provide them 
with rel cased-time training. The employer retains the services of cur- 
rent staff; the employee maintains an income souice; ard the educationa 
institution may be able to use its plant at maximum efficiency. Re- 
leased-time tt'SlTning costs could be passed on as direct costs of serv- 
ice, but, in addition, employees may wish to accelerate their training, 
ancf can be asked to contribute by studying without compensation on 
weekends, holidays, and durin^^ vacation time. 




A strategy to minimize releat ed-time and relief costs starts 
with an overall manpower planning program. With proper planning the 
upper-level "target" job on a ladder is one that will have openings for 
newly trained staff to fill. These openings could come about due to 
new or expanding services, due to turnover, retirement, or due to chronic 
current vacancies. The number of openings to be filled must be knoiro 
before planning can take place. Planning is needed to make sure that 
money is in* the budget for the job titles to be filled when tiie training 
ends and the trainees are ready to work in the titles. 

It is also importa; ' 'hat entry-level jobs -n a ladder be 
able to be filled easily, because some relief workers must be hired at 
this I'^vel. This is not possible if there are no individuals available 
to be recruited and trained to fill the entry-level jobs. 

If the entry level job is one in which employment may be re- 
duced in the futura, then the upgrading program solves tie redundancy 
problem for staff that would otherwise be let go. New staff are not 
needed at entr^ levels, and the cost of upgrading is reduced by the 
amount that would be needed to recruit, train, and employ nev replace- 
ments . 

A multi-stageu, coordinated system of training can fill va- 
cancies and also provide for replacements. Iz involves half-time st^dy 
and full-time income. It should include double-track staging to provide 
training at minimum cost with no loss in production. For the trainees, 
it can provide maintenance of income and job security, while guarantee- 
ing maximum upward mobility. Double-track staging means that two educa- 



tional prograras run simultaneously. Each program is for half the train- 
ees, and runs during the hours that the other half are working. The 
trainees work during non-overlapping time periods; study can overlap 
for weekends* holidays, and vacations. 

The strategy for dovole-track programs is based on the fol- 
lowing consideration?. If trainees work half time and train half time, 
and if relief workers are to be used to maintain output, oiie relief 
worker can relieve two trainees, but onl> if the two trainees are in 
dif ferent time slots. Anything else is a waste of relief worker costs. 
Alternate months, woeks, or days for the tracks are better than alter- 
nate half days, becauL^ half days are wasteful of travel time and the 
warm-up time needed to refocus trainees' attention froTi study t-> work 
and back again. 

^"he multi-stage strategy dv>vetails all the steps In a career 
ladder. With this approach released -time and relief costs can be kept 
below the cost of staffing the jobs whose vacancies are to be filled 
Figure 11 provides a hypothetical example. It shows how dovetailing 
of programs, maximum use of relief wo-^kers, and use of non-overlapping 
trainee work times can keep the co?ts to a minimuui. 

In this example the plan is to fill eight technologist jobs 
(at an institution that is part of a consortium) in the length of time 
needed to train new aides, *:o train aides t^- become technicians, and 
to train technicians to become technologists in a half-time, work/study 
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Figure 11. A MINIMUM COST STRATEGY FOR UPGRADING; STAGED SEQUENC ES 
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^ Assumes that each relief worker reli 
^ Assumes ihat upgrading trainees work 
^ For half the period. 



eves 2 em^^loyees who are each in half- 
half time and study half time al full 
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-time salaries. 
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Figure 11. A MINIMUM COST STRATEGY FOR UPGRADING; STAGED SEQUENCES (continued) 



Stage of Program and Jobs by Level 



en 



3, Upgrading- training of technicians 
begins : 

a. Four aides newly trained as tech- 
nicians are upgraded. 

b. Four technicians relieve 8 techni- 
cians selected for training to be 
technologists. Time required: train- 
ing gap between levels 2 and 3. 

c. Two new hires are trained to be 
aides one quarter way into the 
period (the time required overlaps). 

d. Halfway through the period 2 aides 
relieve 4 aides selected for train- 
ing to be technicians (the time 
required overlaps) . 

e. Three quarters of the way into the 
period another two are hired and 
trained as aides (unless fewer 
aides will now be needed than at 
start). (The time required o^^erlaps 
so that total time is as in a.) 
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44 



Average 
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$19,200 
14,120 
9,960 
$43.280 



4 . Full cycle completed ; 

Eight vacancies filled; 16 staff 
upgraded; 8 new hires. 

Tochnolog^'sts 
Technicians 
Aides 
Total 




20 
12 
12 
44 



20 
12 
12 

44 



12 
4 
4 

20 



8 
8 
8 
24 



20 
12 
12 
44 



or a quarter of the period. 

90 



$27,200 
10,240 
8,280 



program. A ladder in quality assurance leading to radiologic technol- 
""ogist is an example of such a sequence. 

The example assumes that trainees study half-time and receive 
fall-time salaries. Current incumbents, including trainees for upgrad- 
ing, are at maximum salaries for their lines and receive current wages 
until upgraded. New incumbents start at minimum rates. 

Costs are reduced during the program by employing new staff 
only as needed in the staged sequences. All staff used for relief work 
are fully utilized and are retained at the end of the program to fill 
the slots vacated by the staff who have been upgraded. All staff needed 
for reMef work at higher levels are provided from in-house staff. Out- 
put is kept constant. (See the column for full-time equivalent ^imploy- 
ment . ) 

If the 8 technologists were hired from the outside, the to- 
tal salary cost of staffing 44 new employees for one month would be 
$51>000 (or $47,000, depending on whether new technologists would be 
recruited at the top or bottom of their salary range). At the end of 
the training cycle the same staffing of 44 employees would only cost 
$45,720 per month because the upgrading program reduces costs on every 
line where upgrading takes place. The addition-^l cost savings from the 
reduction of training time by using an educational ladder to parallel 
the job ladder and from the elimination of orientation costs are not 
included in the figures given. 



4-26 

92 



Stage 0 in Figure 11 shows current staffing and costs on a 
monthly basis. (The salary figures are illustrative.) In Stage 1 two 
individuals are hired (one quarter of the number of technologist vacan- 
cies). They are trained as aides. 

In Stage 2 the first training step takes place. The new 
aides are able to provide released- time relief for four aides who now 
study to become technicians. Halfway through the period another two 
aides are hired and trained, so that a total of four aides can replace 
the four who become technicians at the end of the training in this 
period. 

In Stage j the second training step takes place. The upgrad- 
ing of four aides to be technicians makes it possible to relieve eight 
technicians to be trained to become technologists. At a point one 
quarter way into the period, Stage 1 is repeated, and then Stage 2, so 
that two new aides again relieve four aides for study. With an addi- 
tional two hired and trained, four new aides are available to replace 
the four aides who are upgraded to be technicians at the end of the pe- 
riod. The training is dovetailed so that a total of eight new techni- 
cians are available through upgrading to replace the eight technicians 
who become technologists at the end of Stage 3. 

At Stage 4 eight vacancies have been filled, sixteen workers 
have been upgraded, and eight new employees have been hired. Sixteen 
jobs formerly filled by staff at the top of their salary range are now 
filled by staff at the bottom of their range. At no time do the costs 
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meet or exceed what costs would have been if the vacancies were filled 
from outside. 

Trainee Failure Costs and Selection Criteria 

0 

Given the need to minimize the costs and time involved in 
training, there is some incentive for the institution to train those in- 
dividuals who are most likely to succeed in the "target job" (the job 
for which the trainees are to be prepared). If, in addition, the exist- 
ence of an upward mobility program can improve the quality of perform- 
ance of individuals in current jobs, the net cost of upgrading programs 
can be substantially reduced. The HSMS approach provides two selection 
criteria that can be assumed to predict trainee success Vecause they 
tie functioning in the current job to function'^ng in the elated tar- 
get job. Assuming that the job ladder reflects an association of tasks 
that require related skills and knowledges, we may conclude tnat the im- 
portant tasks in jobs at varying levels on a job ladder are related. 
The HSMS criteria for trainee selection are as follows: 

1. The current job title from which the trainees should 
be selected for a given target job is the one just 
below the target job on a job ladder, 

2. Thi incumbents within the job ti . from which train- 
ees are to be selected should be those with the best 
ratings for current performance. 

If employees believe that the quality of their performance 

in the current job will be a faccor in trainee *J^election, their current 

performance will be improved; at the same time, the most able trainees 

can be selected. The attractiveness of these criteria is that the 

first one is impersonal; it focuses on all the incumbents in a given 
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job title; the second criterion is reasonable, since it rewards good 
performance. It also reduces the need for testing to performnce test- 
ing or rating of a small group. If performance evaluation is ongoing, 
no additional lesciri^ is vec^uired. 

Another important criterion is that of motivatio n. It is 
a concept which can best be handled indirectly, since it is subjective, 
for the purpose of trainee selection, self-selection for training is 
an acceptable indication of motivation, provided that all employees 
have had adequate access to information about the availability of the 
career mobility training program. 

In any system of upgrading, especially if trade unions are 
involved* the criterion of seniority must also be considered. Senior- 
ity is a perfectly acceptable means of choosing between two otherwise 
equa] candidates, and its use as one among several criteria is compat- 
ible with the HSMS approach.^ 

A Trainee Selection Strat'2gy 

Once the job title of the trainee population has been se- 
lected, the program can be announced. The potential trainee population 
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^ A different sort of criterion is expressed in the practice ailed "cream- 
ing," which involves taking the most educated applicants regardless of 
their current job, "Creaming" is successful in the short run largely be- 
cause educational levels are roughly related to job ladder sequences, ard 
education provides intellectual skills. However, after creaming is over 
and the better educated are chosen, there io then no model foz continued 
selection. Another c-iterion used to select trainees is scores on apti- 
tude tests. The use of aptitude tests is no better than the validity of 
the test used (that is, the extent to vhich the test reflects job content 
and is free of cultural or educational bi^s). The HSMS approach bypasses 
the inadequacies of aptitude tests by going directly to work-reiated cri- 
teria, ^.^'^9 
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would be those in the title who apply for the program, and this limited 
number of staff would be the ones whose current performances are evalu- 



If a program of performance evaluation such as the one pre- 



sented in the next section were underway, the available data might be 
sufficient to select trainees. Otherwise, performance evaluation would 
proceed as follows: 



1. The tasks in the trainee population's job titxe would 
be identified as described earlier in this chapter. 
These would be designated by job level and factor. 

2. Experts, such as supervisors, would select the most 
central tasks in the trainee population's job. These 
would be the reference tasks for the evaluation. 

3. Sup<?rvisors familiar with the applicants' work perfor- 
mance would be selected as raters. 

4. A performance rating instrument would be prepared: 

a. The extended task name for each task selected- 
would be presented. 

b. For each task, the name of the employee to be 
rated and the rater would be entered. 

c. For each task, the rater would be instructed to 
consider the task and the criteria for evaluating 
the outputs of the task or performance of i:he 
tabic. 

d. For each task, the rater would be asked to com- 
pare the given employee's achicjvement of output 
or performance^ criteria with others regularly 
performing the task. 

e. The same scale would be used for each task ^nd 
for each emplovee to be rated. The instructions 
and scale would read rou:;hly as follows: 



ated as a basis for selection. 



Please compare this employee's pexformance of the 
task Tisted above with "he pertormance of other 
persons regtilarly performing this task. Consider 
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the criteria for the output of the task or for per- 
formance of the task, and consider to what degree 
the criteria are met by this employee and by others 
in the same job title. Please check the statement 
that best describes your comparison of this person 
with the others performing this task. 

.Distinctly superior with respect to 
others in title. 

8. . . ( ) . . .Considerably above average with respect 
to others in title. 

).. .Moderately above average with respect 
to others in title. 



6... 



)... Slightly above average with respect 
to others in title. 

)... Average with respect to others in 
title. 



A. . . ( ) . . .Slightly below average with respect 
to others in title. 

3 — ( ).. .Moderately below average with respect 
to others in title. 

).. .Considerably below average with respect 
to others in title. 

).. .Distinctly inferior with respect to 
others in title. 

The scores of each employee being rated would be calcu- 
lated. If an employee is rated by more than one rater, 
scores would be averaged. 

The seniority of applicants would be used to select 
from among applicants with otherwise equal scores. 



Implementation 

An institution committed to upward mobility as a continuous 
part of its manpower function must be aware that this requires planning 
and specification of the means for implementation. Such a program needs 
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careful prior analysis ana^ork if it is to be designed to suit the 
needs of the institution and the needs of individual staff members. 

Implementation of a career mobility approach necessitates 
changes within the instituion such as the coordination of recruitment, 
training plans, and upgrading programs with the operations of the in- 
stitution. Planning and a redirection of focus may be needed. No 
amount of commitment at high management levels can substitute for the 
involvement of middle and lower line personnel in the implementation 
of institutional change. The greatest enemy of a viable mobility pro- 
gram is staff ignorance of what is happening. 

For this reason the issues of upward mobility should be dis- 
cussed at every level in an organization and in cooperation with em- 
ployee organizations where they exist. It should be noted that persons 
are less resistant to upward mobility for others when they have avenues 
open to themselves as well. Thus, a career ladder or lattices linking 
entry-level jobs through graduated sequences to the very highest pro- 
fessional and administrative jobs is most desirable if maximum support 
is to be enlisted. 

EVALUATION OF INSTITUTIONAL PERFORMANCE 

Evaluation is much in the minds of health services delivery 
administrators. There is pressure to review work as a means to greater 
efficiency; more importantly, there is pressure to review work as a way 
to promote quality. This section is a mini-manual for the use of HSMS 
task data in performance evaluation. It shows how the HSMS task de- 
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scriptions oc extended task names can be used to assess whether an in- 
stitution is achieving its goals, to pinpoint the tasks being carried 
out below acceptable levels, or to evaluate an individual's performance. 

Coverage 

The institution must first decide what it wants to evaluate. 
Does it wish to learn whether the institution or department is accomp- 
lishing its goals? Is it to find out how the work in its most import- 
ant functions is being carried o.uf? Is ic to find out how everyone in 
a given job title is carrying out the work assigned? Is it to find out 
how specific individuals, such as new employees, newly trained employees, 
or employees due for a review, are doing? 

If the institution is interested in whether it is accomplish - 
ing its goals, a series of preliminary questions have to be answered at 
the outset. First, the goals themselves must be articulated. Then it 
must be determined how the goals should he manifested in work. The 
mere pronouncement of the objectives or goals of an institution is not 
enough to bring about the performance needed to attain the objectives. 
The institution must be able to point to the means of achieving the 
goals through their embodiment in tasks, elements within tasks, or 
standards of task performance. 

If the institution is interested in the overall functioning 
in a department , it must first know what tasks aro being carried out, 
and then it must decide which of the tasks it wishes to examine, and 
which performers of the tasks it wishes to review. 
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If the institution is interested in examining the vork in a 



given job title it has to know which tasks are being carried out in the 
title c^nd which tasks and performers it wishes to review. Even in re- 
viewing the work of specific staff, it is necessary to know which tasks 
are being carried out b> the performers who are to be reviewed. 

Preparing For Performance Evaluation 

To know whether there are task descriptions to cover all the 
tasks to be reviewed, the administrator would utilize task inventories 
in the manner described earlier in this chapter under the section titled 
"Jsing Task Data To Structure Jobs." That section describes the crea- 
tion of task lists by job title and/or employee name. 

The output of the first step is a set of tasks to be included 
in the review. For each task there should be a list of the names of the 
employees whose performance of the tasks are to be rated, and the name 
of one or more individuals who will rate the performers' work based on 
past experience or by current observation of the individuals at work. 



tomers, other appropriate persons, or a combination of these. In most 
cases the performer's supervisor is an appropriate person to evaluate 
a performer's outputs or task performance because of his or her experi- 
ence or direct observation. However, patients or co-workers might be 
considered for health services tasks. If the output is directly con- 



sumed by the patient, such as when the task is to give personal care, 
the patient may be a reasonable judge of the output. In cases where 



The raters could be supervisors, co-workers, patients, cus- 
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the performer assists a senior co-worker who is not his or her super- 
visor, the co-worker may be the best rater of the task's outputs or 
performance. 

The next step is to edit the task descriptions to reflect 
actual and/or desired in-house performance for evaluation purposes. 
Even though the task descriptions are already written from the point 
of view of approved procedures, the institution may wish to edit these 
to conform to actual practice at the institution and the objectives of 
the review. The institution may wish to address the following ques- 
tions as a basis for refining the task list: 

1. Are the tasks included the most appropriate to ac- 
complish our goals? 

2. In each task, is this the way we want to have the 
task done? 

3. If there are choices of procedures, which do we 
prefer? 

4. If there are choices of equipment, which do we pre- 
fer or have? 

5. What should we be doing that we are not doing? 

The output of this step is a set of task descriptions 
edited to describe the work as the institution requires it to be done. 

Output and Performance Criteria 

Each task description includes a statement naming the out- 
£ut of the task. (It appears in the upper left of the first page of 
the Task Description Sheets.) A task can have a tangible output, such 
as a set of radiographs taken during a particular examination. A task 
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can have an intangible output, such as an explanation to the patient 
of how to prepare at horae for an examination. 

If a task has a tangible output, it should be possible to 
state concretely the criteria for evaluating the quality of the output. 
If these output criteria or standards can be stated explicitly, task 
performance can be evaluated objectively. If a task has intangible 
outputs, it may be hard to state output criteria. This would be the 
case when the output cannot be separated from the procedure, such as 
in giving reassurance, or when largely intellectual processes are in- 
volved, such as in diaj^nosis. In such cases it may be possible to 
state objective criteria for task performance rather than for the out- 
put per se . It may be crucial that all the steps in a task be done 
correctly in a proper sequence. The absence of a step may be as im- 
portant as a wrong step. Such standards can be termed per form ance 
criteria . 

The next step in the evaluation process is to go over the 
tasks to be reviewed and separate those for which objective output cri- 
teria can be written from those for which performance criteria will 
be written. The criteria should then be discussed, written, and re- 
viewed by appropriate expert staff members in the department. 

For a task which requires output criteria , the eventual 
evaluatior instrument will need to contain the task reference and the 
criteria. The extended task name or the output statement on page 1 of 
the Task Description Sheet is probably sufficient as the reference* 
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When a task has several outputs, criteria for all may be vritten or 
the most important output and its criteria can be used. For a task 
which requires performance criteria , the evaluation instrument will 
need the extended task name as the task reference; depending on the per- 
formance criteria, the entire task description or particular elements 
of the task may also be used to highlight performance standards. 

A decision must be made at this stage whether to assess the 
performer's work over a past period of time or to have the raters ob- 
serve the performer during an evaluation period. There are arguments 
for or against either approach. There are negative aspects to relying 
on memory, but there are negative aspects to reliance on a single ex- 
ample when the performer may have been nervous. The practicability of 
observation also has to be considered; some tasks take a great deal of 
time or require that the performer be alone with the patient. The ap- 
proaches may be combined. The decision should be made by the institu- 
tion to suit its particular needs. 

Rating Instruments 

Figure 12, presented later in this section, is a suggested 
performance rating instrument. There would be one such instrunent for 
each task, and as many copies of each as there are raters and performers 
to be reviewed. The sections to be filled in to fit each task are indi- 
cated in biackets. The institution may wish to change the language 
used in this example; however, the instructions to the raters should 
make the following points; 
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!• The rater is to keep in mind only the task named, 
only the criteria mentioned, and only the person 
being evaluated • 

2, For evaluation of past work, the rater's use of the 
scale involves the rater's judgment of whether the 
performer meets the criteria, how many criteria are 
met (if there are several), how often the criteria 
are met in the usual course of the performer's work, 
and the degree to which the criteria are complied 
with. 

3, For evaluation of work being currently observed for 
the purpose of evaluation, all the above considera- 
tions must be eliminated; the work being currently 
evaluated is the only thing that can be considered. 

4. In deciding on the ratings, the rater is to assess 
the performer's outputs or performance using the 
criteria as absolutes, and is not to compare the 
performance or outputs of one performer with those 
of another. 

5. The rater checks a rating value on a nine-point scale 
whose ratings range from highly unacceptable to much 
better than acceptable.^ 

The Rating Data 

The ratings provide data to describe the quality of the task 
performance in statistical terms. They can be expressed as distribu- 
tions of superior or inferior performance around scale point 5, which 
is the minimum acceptable level. The distributions of ratings for each 
task tells the institution about its overall performance cf each task. 
The distribution of ratings for each performer tells the institution 




In the sectior on trainee selection presented earlier in this chapter 
a similar rating scale is presented. It differs with respect to the 
reference. The earlier one compares the performer with others in the 
title. The one here compares the performance with absolute criteria. 
The reason for the difference i^ that one must assume a normal distri- 
bution of ratings for predictive purposes when doing trainee selection. 
When absolute standards are involved, skewed distributions can be ex- 
pected. For evaluation of institutional performance, the skewness of 
the distribution is of interest and, if it is in a positive direction, 
is even desirable. 
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Figure 12. SAMPLE OUTPUT OR PERFORMANCE RATING INSTRUMENT 

p. 1 of 2 

GENERAL INSTRUCTIONS 



You are being asked to consider the work ciCtivities of one 
or more persons employed in this institution. Each work activity, 
called a task, will be summarized for you on one of the following pages, 
along with the name of the person whose work you are to consider. You 
may be asked to consider the work of more than one person doing the 
same task, and/or more than one task done by the same person. However, 
there is a place for you to indicate that you do not feel that you 
have enough information to rate the person or the work represented 
by the task named. 

Your ratings will make an important contribution to deter- 
mining the current general level of performance in the task being rated. 
These ratings can be used to help plan for improvement of work per- 
formance. 

The task statements will each be accompanied by a statement 
of what qualities are considered desirable with respect to the outputs 
which result from the task (the task output criteria), or by a state- 
ment of what performance standards are desirable with respect to how 
the task is carried out (task performance criteria). Several criteria 
may be mentioned. 

If you are asked to consider work carried out over a past 
period of tine, please consider how many of the criteria are met, to 
what degree they are met, and how often they are met by the person 
named, over that period of time. You should then check off the state- 
ment that best describes your evaluation. 

If you are asked to evaluate work you are currently observ- 
ing, please consider how many of the criteria are met and the degree 
to which they are met only in the work you are currently observing. 
Then check off the statement that best describes your evaluation. 

Try to be fair, objective, and impartial in your ratings. 
Base your ratings on the employee's attainment of the criteria for the 
task named and not on any personal characteristics which he or she may 
have; do not compare this person's performance or outputs with those 
of others. Please do not let your evaluation of this person's perform- 
ance in one task affect your judgment of how another task is done by 
the same person. 

Please fill in your name and title wherever it is called for. 
Thank you very much for your cooperation. 
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Figure 12. SAMPLE OUTPUT OR PERFORMANCE RATING INSTRUMENT (continued) 

p. 2 of 2 



OUTPUT OR PERFORMANCE RATING SHEET 



1. Please enter: Your Name Title 



2, You are asked to consider the following task: (Code No, ) 

(Institution fills in the task code number 
(above) and in this space enters the ex- 
tended task name. Institution may also 
wish to name the task output (s),] 

3. You are asked to consider employee: (Name) 

(Title) 

^. Do you feel qualified to evaluate the quality of this person's work 
performance in this task? ,.. ( ) Yes; (No). If no, please go 

on to the next task. If yes, please go on to item 5. 

5. Please consider the criteria to use to evaluate the task's output or 
the performance of this task: 

[Institution fills in the output or per- 
formance criteria. ] 



You are being asked to evaluate the performance or output of this 
employee in one of the following ways: 

( ) Consider the employee's work over a substantial and representa- 
tive period of time; do not concentrate on very recent experi- 
ences or only on outstanding examples of achievement or failure, 

( ) Observe the employee at work currently. 

[institution selects one or both, J 

Please rate this employee according to your judgment of the degree 
to which he or she meets this task's output or performance criteria: 
How many criteria are met, and how well are they met? Please check 
the statement that best describes your evaluation. 



erJc 



9. 
8. 
7. 
6. 
5. , 
4. , 
3. , 
2.. 
1. . 



.Distinctly superior with respect to criteria. 
.Considerably above acceptable with respect to criteria. 
.Moderately above acceptable with respect to criteria. 
.Slightly above acceptable with respect to criteria. 
.Acceptable with respect to criteria. 
.Slightly below acceptable with respect to criteria. 
.Moderately below acceptable with respect to criteria. 
• Considerably below acceptable with resp-sct to criteria. 
.Distinctly inferior with respect to criL->ria. 
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about the competence of individuals. It is then possible to pinpoint 
problem tasks and problem performers and design remediation through 
training or reorganization. 



lected, these should be arranged in sets by task, and arranged within 
each task set by order ol the scale value checked. The results can 
then be entered in a table similar to that presented in Figure 13. On 
each row a task^s code number and a very abbreviated task name are en- 
tered. Column (1) is the total number of ratings for a task. This 
would be equal to the total number of performers being rated on the 
task. (If performers receive ratings by several raters it might be ne- 
cessary to first average all the ratings for a given performer.) In 
column (2) there are sub-columns, one for each scaJe value. For each 
task, the number of ratings at each scale value is entered. (Their sum 
should equal the figure in column (1).) Column (3) is the percentage 
distribution of the scale values. It is necessary to do percentage 
distribution so that comparisons can be nade from task to task. (The 
percentagp distribution is obtained by dividing a given entry in a sub- 
coluriin in (2) by the total figure in column (1), and multiplying by 100.) 



ployees. In Figure 1^, each row refers to an employee. Column (1) is 
the total number of tasks on which the employee has been rated. Col- 
umns (2) and (3) now refer to the distribution across employees. 

The institution is now in a position to judge which tasks are 

being performed at acceptable levels, and to what extent. Ratings at 
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When the Output or Performance Rating Sheets have been col- 



A similar table can be irade that shows the ratings for em- 
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Figure 13. SAMPLE HEADING FOR TABLE OF OUTPUT OR PERFORMANCE RATINGS BY TASK 







Total 


i Ratings of Performance by Task 




Task 


Task 
Rat- 


iNumber of Output or Perfor- 
jmance Ratings by Scale Value 




Percentage Distribution of 
Ratings by Scale Value 


Task Name (Abbreviated) 


Code 


ings 


1 9 


8 1 7 


6 


5 


4 


1 31 2 


1 1 




9 


8 


7 6 


5 




3 


2 ll 


No. 


(1) 


(2) i 






(3) 










I 

t 

1 










^ 










1 
1 

i 
1 

1 

1 













Figure 14 > SAMPLE HEADING FOR TABLE OF OUTPUT OR PERFORMANCE RATINGS BY EMPLOYEE 



Employee Name and Job Titj_e^ 



Total 
Empl . 
Task 
Rat- 
ings 



111 



Ratings of Performance by Employee 



Number of Output or Perfor 
ma nee R atings by Scale Wihie 



-12) 



Percentage Distribution of 
Ratings b y Scale Value. 



9|8| 7|6|5|4| j| 2 



ri- 



scale points 1 to 4 fall below acceptable levels. Ratings from 5 to 
9 are at or above acceptable levels. The institution can now decide 
uhat level of achievement It wishes to attain, and what type of dist 
bution for a task warrants the task being considered a "problem task." 

Figure 15 represents a graphic portrayal of a hypothetical 
distribution based on column (3) of Figure 13. The distribution re- 
presents hypothetical data for thirteen tasks whose code numbers are 
listed along the bottom horizontal axis. 

Each bar representc a task. Each vertical division on the 
left-hand vertical scale represents ten percentage points. Within each 
bar, the percentage distribution of the ratings for a task are laid off 
by scale value. Rating scale values starting from the value of 5 are 
laid out above the zero line and move up to 9; the scale value of 4 is 
laid out below the zero line; others follow down to 1. The number that 
appears in an area within a bar indicates the scale value represented 
by the area in which the number is found. Laid oat this way, the area 
above the zero line shows the percentages of the tasks ac acceptable 
ranges, and the area below the zero line shows the percentages of the 
tasks at unacceptable ranges. 

The institution can now see that a task such as Task 6, 
while having a 40 percent distribution below the "acceptable" level, 
has none below the rating of 4, "slightly below acceptable." Any task 
with as much as ten percent of its ratings at 1, "distinctly inferior," 
might be in trouble. Task 5 is such a task. Tasks 6 and 8 have no 
ratings above 6, "slightly above acceptable," and that might be of 
Q 4-43 
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Figure 15. HYPOTHETICAL GRAPHIC REPRESENTATION OF DISTRIBUTION 
OF OUTPUT OR PERFORMANCE RATINGS BY TASK 
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Each bar represents distribution within one task. 
Numbers within bars represent the scale value which 
the area represents. Areas are percentages as read 
on the vertical scale. 
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concern. The clearest problem task is Task 9, with 70 percent of its 
outputs ''moderately below acceptable'* or worse,, and none better than 
"acceptable/* 

A similar visual presentation can be prepared for the em- 
ployee data. In such a case the bars would refer to employees rather 
Chan tasks, and the distributions would indicate performance ratings 
across the tasks of the performer's job. Once the "problem tasks" or 
"problem employees" are located, it becomes possible to diagnose what 
it is about the quality of the performance or of the output that has 
given rise to the inadequacy of the results. Then remediation can be 
planned . 
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APPENDIX A 

TECHNICAL HISTORY OF THE HEALTH 
SERVICES MOBILITY STUDY INSTRUMENTS 

The following pages are excerpts from an earlier method man- 
ual developed by the Health Services Mobility Study (HSMS) in 1971 / 
Since then HSMS has further developed and refined its instruments and 
methods. This appendix reports on the work that was done to develop 
the analytic instruments and to assure their reliability. 

CONCEPTS 

Reliability is the consistency with which an instrument yields 
the same results when a set of observations are made by different people 
or are repeated over time. An instrument such as a task definition is 
of limited usefulness if it yields different results when applied one day 
than it does when applied another, or if different results are obtained 
when two or more different analysts apply it to the same observations. 
The relevant aspect of reliability for task identification is inter-rater 
reliability. This refers to the consistency with which two or more indi- 
viduals using the same definition identify the same tasks for the same 
performer. 



an- 



A Job Analyses Method For Developing Job Ladders And For Manpower PI 
n±n^, Health Services Mobility Study, Research Report No. 3, Part A, 
Volume III, ''Task Identification: Data.*' Part B, Volume III, ''Skill 
Dimensions of Tasks: Data." and Part C, Volume III, ''Knowledge Iden- 
tification and Scaling of Tasks: Clustering Tasks." 1971. This docu 
ment is no longer available for distribution. It has been replaced by 
Research Report No. 11. 

ERIC ll4 



Validity is a difficult concept with which to deal, both 
conceptually and practically. The validity of an instrument is con- 
cerned with what the instrument identifies and how well it identifies. 
It deals with the extent to which an instrument identifies what it is 
supposed to identify. 

The relevant aspect of validity for task identification is 
content validity . This involves the systematic examination of the con- 
tent of the data to determine whether they cover an accurate and repre- 
sentative sample of the activity to be identified, 

THE DEFINITION OF TASK 

HSMS revised its definition of "task" during a series of 
three field tests (referred to as pre-teot, re-test, and pilot test). 
This section describes the HSMS field testing of its definition of 
"task*' and the results. 

Reliability 

The HSMS field tests measured inter-rater reliability in 

task identification. The tests measured the extent to which several 

analysts, each exposed to the same training, the same situations, and 

the same experiences with performers, identified the same tasks, A 

given number of nerformers were observed and interviewed by a group of 

HSMS analysts who then independently filled out task identification 

data. Reliability was measured in terms of the analysts' agreement on 

the output, what is used, and the recipient, respondent, and co-workers 

for a task, with full agreement including all three components, 
Q - A-2 
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An overall reliability score and a score for ^ach anal yst 
provided information about the method itself as well as about the func- 
tioning of each analyst. We found that the reliability of the analyst 
improves with practice; thus the user can expect increasing relial.ility 
over time* 

HSMS devised a Standard Reliability Scor e vhich provides a 
standardized measure that can be compared across test situations. The 
Standard Reliability Score for any analyst measures his agreement with 
other analysts. The formula for the Standard Reliability Score is as 
follows : 



Aj. = W , where: 

(N-1)(T) 

A^ = the Standard R.^t lability Score for an analyst, per 
performer; 

W = the relevant summed agreement scores (the sum of the 
number of other analysts in agreement with a given 
analyst on the identification of each of the tasks 
identified by any and all of the analysts);^ 

N = the number of analysts involved in the test; 

T = the total number of separate tasks identified by all 
the analysts in the team for the performer. 



A^ is taken to two decimal places. Perfect inter-rater re- 
liability for an analyst would be 1.00. For example, if five analysts 
in a test agreed on the same 20 tasks for a particular performer, the 
summed agreement score for any one analyst would be the sum of 20 tasks. 



2 

The summed agreement scores cannot by themselves provide standard mea- 
sures of reliability because their absolute values are affected by the 
number of analysts and the total number of tasks identified in a parti- 
cular test. 
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times the agreement score of 4 for each, or 80. Tne Standard Reliabil- 
ity Score would be: 80 , or 1.00. The analyst's inter-rater 

(4) (20) 

reliability is his average across performers; overall reliability is the 
average across analysts. 

Reliability Test Results 

The pilot test was conducted under very difficult circum- 
stances. Because of time pressures and Illnesses, the reliability data 
were collected for three performers, rather than for the five origi- 
nally planned. In addition, the analysts went immediately from class- 
room training in the revised method to field testing without prior eval- 
uation of their field performance. As a result, the testing for some of 
the analysts may have reflected the reliability of the definition as it 
was in the second test. (Misconceptions were clarified after the test 
data were collected.) 

Statistical Data 

Table 1 shows the analysts' reliability scores in the three 
tests. For the pilot test, separate scores are also shown for tasks of 
each of three performers. Colutnn headings refer to the tests and to the 
performers, respectively, moving from left to right. There were five 
analysts in each test; only one analyst (Analyst 1) was involved in all 
three tests; one other analyst (Analyst 2) was involved in the last two 
tests. Any analyst involved in the first or second test was reported 
on only if he or she was present for the third test. 
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TABLE 1. COMPARISON OF TASK ID RELIABILITY SCORES 







Test 


Situations'^ 


Pilot 


Test Performers'^ 




Analyst 


Pre- 


Re- 


Pilot 


EKG 


Family 
Health 


Nurse 
Practi- 




test 


test 


Test 


Tech. 


Worker 


tioner 


(1) 


No. of Performers 


: 5 


11 


3 




(2) 


Average^ 


.55 


.57 


.53 


.51 


.53 


.55 


(3) 


(T) 1 


.55 


.55 


.56 


.54 


.56 


.59 


(4) 


(S) 2 


n.a. 


.57 


.55 


.53 


.55 


.57 


(5) 


(F) 3 


n.a. 


n.a. 


.53 


.55 


.52 


.53 


(6) 


(M) 4 


n.a. 


n.a. 


.50 


.52 


.47 


.52 


(7) 


(L) 5 


n.a. 


n.a. 


.49 


.40 


.53 


.55 



n.a. Not applicable. Analyst was not on staff at the time. 

^ Average based on five analysts for each test; analysts varied f 

test to test. 
^ In order as studied, from left to right. 
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Line (1) indicates the number of performers studied in each 
test situation; there were five, eleven, and three, respectively, in the 
three tests. The three performers for the pilot test are listed in in- 
creasing order of job level. This was expected to provide a rising level 
of difficulty for the analysts, since difficulties in determining task 
boundaries generally increase as the job level rises. The performers 
were studied in the order shown. Since reliability tends to increase with 
practice, we expected an offsetting effect to the increasing difficulty. 

Line (2) presents the overall reliability scores. There was 
some falling off from the second test. Pilot test reliability was .53. 
This is probably the result of the inexperience of the analysts in the 
test. As line (3) indicates, Analyst 1, present for all three tests, 
has scores that rise from test to test. Analyst 2's score dropped 
somewhat from the re- test to the pilot test. However, both of these 
analysts had prior experience and scored higher than the three without 
prior experience. 




The effect of experience is also manifest when one examines 
the separate averages for the three performers of the pilot test (line 
2), There is a rising trend, despite the rising level of complexity of 
the jobs. 

Interpretation 

The pilot test results indicate that careful training and 
field experience are needed to raise reliability above an average of 
•53, However, ,53 is an acceptable figure for reliability when one 
considers that we are not dealing with probability samples. The figure 
suggests that, with a small degree of training, half the analysts will 
always agree, or that all the analysts will agree half the time ~ with 
all the possible variations between. 

Validity 

In the first two tests HSMS used the concept of ''accuracy*' 
as an approximation of validity. We measured the individual analyst's 
agreement with an analyst on the team who was designated as the "expert 
analyst,*' The expert was the senior staff member who trained the others 
and who was presumed to be better able to use the method. The pilot 
test was to have departed from the use cf the '*expert*'; the accuracy 
score was to have been a measure of congruity with the group judgment . 
It was as£:nmed that the solutions arrived at through discussions in a 
team Task ID Conference would be better than those of a single analyst. 

In the first two tests, the sum of "correct" identifications 
was not used as a measure of accuracy, since the sum could not also re- 
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fleet excessive task identification as errors. Our Standard Accuracy 
Score adjusted for excess identifications and also permitted comparison 
of accuracy measures across test situations, even when the correct num- 
ber of tasks varied. The formula for the Standard Accuracy Score is 
as follows: 

A^ = , where: 
T 

A^ = the Standard Accuracy Score for an analyst, per performer; 

C = the number of relevant correct answers: the total num- 
ber of tasks correctly identified, minus the total num- 
ber of tasks identified by the analyst in excess of the 
correct total. (If the result is a negative number, it 
is entered as a zero.); 

T = the total number of correct tasks (as per criterion). 

On further reflection, HSMS concluded that a good statistical 
measure of validity was not available since, when the analysts are newly 
trained the conference or group decisions are no better than the group's 
overall mastery of the method, and the "expertise" of any given senior 
analyst cannot be measured objectively. 

Rather than test for validity, HSMS built procedures into the 
method to ensure that the tasks identified would conform to the defini- 
tion and adequately reflect the performer's work. These include the re- 
quirements that there be team agreement on task identification, the use 
of literature, and review by the performer, the director, and by a mini- 
mum of three expert reviewers. These procedures provide content validity 
which is verified by inspection. 
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S KILL SCALES 

Development of Several Skill Concepts 



In dealing with interpersonal i n^ ion skills , the HSMS 



staff members provided seve*.'al important insights. They suggested that 
therapeutic interaction is no different from non-therapeutic interaction 
with respect to skill, but, rather, differs only with respect to motiva- 
tion. For that reason, four original interpersonal interaction skill 
seal s (Therapeutic, Non-therapeutic, Co-worker Cooperation and Leader- 
ship) were combined into two skill scales (Human Interaction and Leader- 
ship) . The Leadership skill reflects the general literature on the con- 
ditions of leadership, rather than the nature of leadership. Both of the 
interpersonal skills are described in terms of the conditions under which 
they are required to be exercised rather than their manner of expression. 
This allows for individual differences in the way in which the skills ure 
manifested according to personality differences in the performers. 



understood that the Knowledge System woald deal with vocabulary and gr^^m- 
mar. The three scales relate to precis ion in th« choice or use of lan- 
guage. Originally, there were four lan^iuage scales; testing disclosed 
a high correlation between Comprehension of Spoken Language and Oral Use 
of Language, and we decided to combine the scales. Such combination was 
not found to be warranted with Readin^ and Writing. It is logical to 
expect that these latter skills are needed independently. 

HSMS had long considered the need to account for and measure 
intellectual skills as well as manual and interpersonal skills. We 



The content of the language skills was clarified once it was 
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first conceived of one skill, General Intellectual Development, It was 
to contrast with General Educational Development, and refer to an intel- 
lectual capability that develops as knowledge is attained, but is inde- 
pendent of specific knowledge content* It soon became clear that we 
were dealing with a multidimensional skill concept. 



ber of skills that would account for intellectual skills of a general na— 
ture that would be learnable, free of specific content, and scalable with 
respect to task requirements. 

Using the work of J,P, Guilford as a departure point, ^ we 
workec with three broad areas of mental content: Semantic, Symbolic, and 
Figural, and, within each, three broad areas of mental activities : Cog- 
nition, Mental Search, and Evaluation, We also identified and included 
four specific mental relationships ; Classification Principles, Changes 
in Mental Content, Organizational Principles, and Implications, The 
various combinations of these dimensions resulted in 36 scales. 

To find how the scales might *Wk" in the field, we decided 
to sample a variety of jobs by function and by level of educational re- 
quirements. We created a set of six functional and six educational cri- 
teria to select performers and job titles. This resulted in a matrix 
of 36 job category cells. We then attempted to find two performers for 
each cell, one to be interviewed only and one to be observed and inter- 



J,P, Guilford, The Nature of Human Intelligence , New York: McGraw- 
Hill Book Co, , 1967, 



HSMS designed a crash program to determine the nature and num- 
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viewed. The design called for identification and scaling of three tasks 
per performer, with the tasks selected to be representative of the job, 
without duplicating tasks. This gave us a set of 216 tasks. 



required in task performance. The field manual described the CIS skills 
and presented a definition, a description of the principles to use in 
scaling, and a set of questions which could be asked aloud by the ana- 
lysts or posed tc themselves to aid in scaling. (We used a numerical 
scale . ) 

The scaling data were punched on cards, one task per card, 
with accompanying identifying information. The data were then evaluated 
in a series of runs using principal component factor analysis. 



lectual skills. We concluded that different intellectual skills are 
probably invox ^d when the material acted upon or worked with is seman- 
tic, symbolic, or figural in content. 

Identifying tb 3 nature of the three CIS content skills was 
a problem, however. We could not deal with semantic, symbolic and figu- 
ral skills without clarification. To be iearnable through practice and 
scalable, the behavior involved must be more concretely specified. It 
had been difficult for Khe analysts to discriminate among tb twelve 
content skills that clustered on each content factor; it was hard to 
scale independently for each. 



We trained our analysts to scale for only those GIS skills 



We 



found that content was the chief discriminator of intel- 
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We decided to separately factor the variables that clustered 



on each original factor; these were the variables for each of the three 
content areas (semantic, symbolic and figural) covering mental act-»'vities 
and relationships. We also reasoned that factor structures might be most 
differentiated if we factored only tasks collected from the upper half 
of the data" base wi h respect to the educational level required for the 
jobs from which the tasks were drawn. We thus then used the data from 
108 of the 216 tasks, running variables from each of the three content 
area factors separately. The results were hs follows: 



1. The factor structures within each of the three con- 
tent areas were different. Only in semantic skills 
was the structure obtained in the various runs con- 
*5istent and robust enough to suggest discrete vari- 
ibles within semantic skills. 

2. In semantic skills there appeared to be four factors. 
When high inter-correlations were discounted, and 
relevance to use in task analysis was considered, 

two factors remained. One was related to iippli ca- 
tions and the other to class characteristics and 
organizing principles . The two which were discarded 
included one dealing with changes in semantic mean- 
ing and one dealing with cognition, neither of which 
was considered relevant to our needs. 

3. Symbolic skills and figural skills had less stable, 
less consistent factor structures. We decided that a 
single skill could adequately cover each content area. 

A. The factors dealing with semantic implications and 
semantic class characteristics and organizing prin- 
ciples covered mental relationships . We reasoned 
that, as such, they might apply across the three con- 
tent areas. Thus, we arrived ac four General Intel- 
lectual Skills: (1) Implicative Skills; (2) Taxo- 
nomic Skills Tclass characteristics and organizing 
principles); (3) Symbolic Skills; and (4) Figural 
Skills. 

5. HSMS scaling principles for each skill were subse- 
quently worked out by staff members. 
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Statistical, Attributes 

The attributes of sc alabil ity and unidimensionalit y were made 
possible for the scales by determining ai describing the nature of each 
scalers quality, naming the scale appropriately, and identifying the 
scaling criteria. The scaling criteria are called scaling principles . 

In the final revisions of the scales, scaling principle(s) 
were specified for each skill. In cases where more than one principle 
was necessary, a description of three levels for each principle was 
formulated. Then all cohibinations of the levels for the principles were 
examined to see whether they made sense; that is, we asked the question, 
"Can we conceive of a task displaying this combination of levels of the 
principles?" A set of tentative descriptors were then written. 

Equal Appearing Intervals 

The attribute of reliable scale values was achieved through 
the Thurstone technique of "equal appearing intervals." 

The idea underlying the technique is that a scale can be 
made up of descriptors, but values cannot be arbitrarily assigned to 
these, because the real or perceived "distance" between the descriptors 
along the scale must be a*.curate for later applications of multivariate 
statistical techniques. The problem is resolved by asking independent 
raters (judges) to assign values to a set of descriptors arranged at 
random. Scale vali.es are calculated using the median scale value of 
the judges* responses. The reliability of the median is assesed by 

125 



determining the interquartile spread between the 25th percentile and 
the 75th percentile values. (The median is at the 50th percentile). 
Usually, a spread of 2 points or more is considered unreliable, and 
.»«_«:^n$xb, a descriptor is discarded. 

The formula for the median scale value is: L 4- (50.00) -^P^) 

p ~ 

where, w 

L = lower limit of interval in which the median falls 
(50th percentile). 

= percentage of cases that fall in the interval con- 
taining the median. 

^P^ = cumulative percentage in the interval below the one 
containing the median. 

On a nine-point scale (which we used) there are ten intervals. 
The first contains the zero. The frequen cy of a descriptive item is 
calculated by counting the number of judges that placed the item in a 
given interval. The percentage referred to above is the frequency of 
an item in an interval, as a percentage of the number of judges whose 
responses are being used. The cumulative percentage referred to above 
is the percentages of the item in a given interval, plus the sum of the 
percentages of the item in all lower intervals. An example of our 
equal interval tests is presented in Figure 1. 

The scales have gone through a number of revisions. Each 
time a scale was revised it was subjected to another equal interval 
test unless the revision was only a minor language change. Table 2 re- 
ports the results of the equal interval tests for the 18 HSMS scales 
in use [in 1971]. 
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^^Sure 1. EQUAL INTERVAL TEST >;r uiNG LEVELS OF LOCOMOTION 

Instructions to Judges 

ncoH ^. A ^^^"8 "-^^^^ ^° "te a set of statements which may be 

used to describe the levels of a particular skill. The skill is one 
whxch can be called for in the performance of job tasks. The skiH to 
be considered here is Locomotion. 

of . n.r.f/^^^ f^^^ "f^""^ '° '^^Sree of body coordination required 
of a performer in a task. The skill involves the movement of tlie per- 
former s body torso or limbs through space in order to achieve prede- 
termined standards of body movement or position. 

The level of the skill rises with the degree of body coordina- 
i^olvT" ;h determined by the complexity of the standards 

involved, or the complexity of the external circu„,stances which restrict 

^^^^'^^^^ l^^^l is not determined by considerations of 
strength or the level of knowledge which may be involved. 

u 1?".''^^ following pages you will find a nuirber of statements 

leries'of t'"')' 'T'' °J ^^^^ ^° statemenri a 

tion 9 is"rn ° J° ^' Q is ^° represen t the absence of T.n.nn.n - 

tlon 9 IS to represent the highest possib le level of Lo comor^nn .. h.m T 
a task can require . You are asked to check off a va lue for each state- 
in 6n L • " 

Please read the entire set of statements first, keeping in 
mind that you are asked to rate each statement with a number from 0 to 
9. according to the level of Locomotion which the statement represents 

iLrT.. A "^^'^ ^ g^i'l^- The statements have 

been lettered at random. 

You are asked to check a value for each statement so that 
tZ.rT l^'^"" r' ascending order. The statements which you con- 
sider to be at higher levels must be assigned values higher than state- 
ments which you consider to be at lower levels of the skill. 

You need not assign a ll the numbers from 0 t o 9. * l-f you think 
that two statements are at an equal lev e l they shouldT7~.Pn rp H%h. ..r,^ 
even if they do not have the same worH.np. Equal numerical difflr Pn.r 
should re present equal degrees of the skill . 

Once you have sorted the statements to your satisfaction, 
please reread each statement and check off the scale number which, in 
your opinion best represents its level of Locomotion. Remember that 0 
is to be used when the skill is absent or irrelevant, and 9 represents 
the highest possible level of the skill. represents 

Please see that you have properly checked the scale points 
which you have selected for each statement. Be sure that every state- 
ment has one and only one scale point checked. Please write your name 
Deiow. Thank you for your cooperation. 

Judge's Name 
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Figure !♦ (continued) 2 of 2 

No Highest level 

Locomotion pf rhi> oir^n 

. . . T 

-4 -I . 1 1 > 



t 



^ 1 1 H 



0 12 3 4 
Descriptive Statement Scale: Locomotion 



(a) The task requires the performer 
to move his body, torso' or limbs 

through space so as to achieve (check one) 

somewhat complex, predetermined 0123456789 
standards for body motion or . . | . | i | j 

position. A moderate degree of ' 

body coordination Is called for. ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) 



(b) The task requires the performer 
to move his body, torso or limbs 

through space so as to achieve (check one) 

extremely complex^predetermlned 0 12 3 4 5 
standards for body motion or 
position. An extremely high 




I i H h 



degree of body coordination Is ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) 
called for. 



(c) The task does liot require the 
performer to move his body, 

torso or limbs through space so (check one) 

as to achieve a predetermined 0123456789 
standard for body motion or | ] j | | ^ | | | [ 



position. 



( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) 



(d) The task requires the performer 
to move his body, torso or limbs 
through space so as to achieve 

simple, predetermined standard(s) (check one) 

for body motion or position. 0123456789 
A small degree of body coordl- . . . , . , 

nation Is called for. * ^ ' ' ' ' ' » ' 

( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) 

(e) The task requires the performer 
to move his body, torso or limbs 

through space so as to achieve (check one) 

considerably complex, predeter- 0123456789 
mined standards for body motion . . * . . . 

or position. A high degree of ' ^ ' ' ' ' 

body coordination Is called for. ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) 
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Table 2. DATA BASE FOR SCALE VALUES USING THURSTONE EQUAL INTERVAL SCALING 

(p. 1 of 2) 



and Number of Judges 


Description 


a b 


c 


d 


e 


f 


g 


h 


i 


j 


1. Frequency 


Median Scale Value^ 


5.0* 2.0* 


8.0* 


0.0* 


3.0* 


9.0* 


1.0* 


7.0* 


4.0* 




(15) 


Interquartile Range 


1.0 0.9 


0.8 


0.5 


1.1 


0.6 


0.8 


0.7 


0.1 


























2. Locomotion 


Median Scale Value^ 


5.0* 9.0* 


0.0* 


1.5* 


7.0* 












(15) 


Interquartile Range 


1.9 i 0.6 


0.5 


1.0 


0.9 


































3. Object Manipula- 


Median Scale Value^ 


1.5*' 0.0* 


5.0* 


9.0* 


6.5/ 


3.5* 


7.5* 


3.0/ 






tion (15) 


Interquartile Range 


1.0 . 0.5 


0.9 


0.6 


1.4 


1.2 


1.0 


0.9 






- 






















4. Guiding or Steer- 


Median Scale Value^ 


1.5* 4.0/ 


9.0* 


7.0* 


0.0* 


3.0* 


5.5* 


8.0/ 






ing (15) 


Interquartile Range 


1.1 ' 1.2 


0.4 


1.0 


0.5 


1.4 


1.0 


0.9 




























5. Human Interaction 


Median Scale Value^ 


5.0* 7.0* 


3.0* 


0.0* 


9.0* 


1.0* 










(18) 


Interquartile Range 


1.1 ' 1.0 


1.5 


0.0 


0.6 


0.8 














1 ■ 


















6. Leadership 


Median Scale Value^ 


4.5//i 4.0/ 


1.0// 


8.5// 


5.5/ 


3.5/ 


6.5// 


5.5/ 


3.0// 


0.0* 


(22) 


Interquartile Range 


1.5 2.0 


0.7 


2.1 


3.1 


1.4 


2.1 


1.6 


1.2 


0.4 


























7. Oral Use of 


Median Scale Value^ 


4.0* 


7.5* 


2.0* 


0.0* 


9.0* 












Language (18) 


Interquartile Range 


1.2 


1.1 


1.0 


0.5 


0.6 




































8. Reading Use of 


Median Scale Value^ 


2.0* 


7.0// 


0.0* 


9.0* 


5.0* 












Language (17) 


Interquartile Range 


1.2 


0.9 


0.5 


0.8 


1.6 




































9. Written Use of 


Median Scale Value^ 


5.5* 


5.0* 


9.0* 


2.0* 


0.0* 












Language (15) Interquartile Ranpe 


1.1 


1.1 


0.8 


0.8 


0.5 













* Item was kept. 

// Item was edited. 

/ Item was eliminated. 



Rounded, 
b Refers to 



number of judges in equal intervaJ test. 



Table 2. 



DATA BASE FOR SCALE VALUES U S ING THURSTONE EQUAL INTFRVAL SCAT.TNH (continued) 

(p. 2 of 2) 



Scale Name, Number; 
and Number of Judges^ 



10. Decision Making 



on Methods (17) 



11. Decision Making 



on Quality (16) 



12. Figural Skills 
(15) 



13. Symbolic 



Skills 



(16) 



14. Taxonomic 
Skills 



(15) 



15. Implicative 



Skills 



(15) 



16. Financial Conseq. 



of Error 



(15) 



17. Conseq. of Error 



to Humans (15) 



18. Levels of Know- 



ledgf 



Statistical 
Description 



Median Scale Value^ 



Interquartile Ran^e 



Median Scale Value^ 



Interquartile Ran^e 



Median Sea le Value^ 



Interquartile Range 



Median Scale Value^ 



Interquartile Range 



Median Scale Value^ 



Interquartile Range 



Median Scale Value^ 



Interquartile Range 



Median Scale Value^ 



Interquartile Range 



Median Scale Value^ 



Interquartile Rang e 



Median Scale Value^ 



Items Listed in Order Presented to Judges 



6.0/ 



2.0 



7.0* 



1.1 



5.0// 



0.9 1.5 



3.0* 



1.6 



1.5* 



1.3 



6.0/ 



1.5* 



1.3 



7.0* 



5.0* 



0.7 



4.5* 



1.5 



9.0* 



0.4 



9.0// 



0.4 



9.0* 



0.5 



0.0* 



0.5 



5.5* 



1.4 



3.0/ 



1.3 



7.0* 



1.5 



2.0* 9 . 0* T75* 



0.9 1.5 0.4 



3.5/15.0* 



1.2 10.9 



6.0*, 1.0* 



0.7 !0.8 



J- 



5.5*: 2.0* 



2.0* 



0.8 



1.9 



9.0* 



0.4 



9.0// 1 4.0* 



l.r II. 1 



1.0 |0.7 



4.5/^175* 



0.0*1 7.0* 



0.5 T0T6 



5.5* 



3.5* 



7.0* 



1.4 



0.0* 



0.5 



0.0* 



0.5 



7.5/ 



1.0 



0.0* 



0.5 



6.5/ 



1.5* 



0.9 



3.5* 



1.5 



8.0/ 



0.9 



<?.0* 



0.6 



6.5/ 



1.2 



3.5// 
1.2 



4.0/ 



1.3 



1.0 



0.0* 



0.5 



9.0* 



0.4 



0.0* 



0.5 



7 . 5'- 



1.1 



1.0* 



0.8 



0.0* 



7.0* 



* Item was kept. 
// Item was edited. 
/ Ite^was eliminated. 



0.0* 



0.5 



4.0/ 



3.7 



2.0* 



0.9 



.0// 



2.5/ 



1.2 



0.6 1.5 



3.5* 



1.7 



1.0* 



0.8 



8.0* 



0.5 



4.0* 



1.1 



8.0* 



0.8 



9.0* 



0,3 



— — _ , — -■ --- .- .^ .^.^ , ■ , .v.-, 9.0*i 2. 5* o.u 

Interquartile Range | 1.4 |1.0 ll.4 |l.4 |o.6 I 0.7 |o.6 |l.6 lo.8' 



^ Rounded. 

^ Refers to number of judges in equal interval test. 
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The column to the far left presents the scale name and num- 
ber and indicates in parentheses the number of judges used. The next 
column from the left provides a line for the rounded median scale \ui\ue 
for an item and a line for the interquartile range. Columns (a) through 
(j) represent the items in random ordei , as they were presented. A 
median scale value marked with an asterisk (-) indicates an item kepi 
for the scale. A number sign (//) indicates an item kept but revised. 
A slash mark (/) indicates that an item was discarded. 

Items were discarded for one of three reasons: 

1. The item had an unacceptably high interquartile value. 

2. The item was too close in value to another acceptable 
item to be considered to bp ai.-^-'nfiuishable in the 
field. 

3. Inclusion of the item would not permit the scale to 
be self-evidently cumulative. 

Cumulat iveness 

Originally, we tested each scale to make sure that its items 
were cumulative. This was done by having judges indicate which other 
descriptors on a scale (presented at random) could be assumed to also 
be contained within the level represented by the given descriptor. The 
count for inclusions and exclusions was evaluated using probability 
theory to determine when there were significarit indications of cunuLi- 
tiveness for a given descriptor. The latest scales, however, are based 
on selectively ascending descriptors. By eliminating items whi^h drop 
on any scale principle as the scale is ascended, the cumulativeness of 
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the scale becomes self-evident. (Since the Decision Making on Quality 
scale is an exception, it was tested statistically for cumulativeness . ) 

Reliability and Accuracy Testing 



result of their being field tested for reliability and accuracy (con- 
formity with a criterion for the correct scale values). There were 
three field tests. 

In HSMS tests "reliability" is defined operationally. It 
means the extent to which several analysts, each exposed to the same 
training, the same situations, and the same experience with the per- 
former, agree in their scaling of a given set of tasks. This is a mea- 
sure of inter-rater reliability .^ Our procedure involved calculation 
of an overall score for each scale and a score for each analyst. It 
thus provided information about each scale as well as about the func- 
tioning of each analyst. 

The calculation was applied for each scale, using only those 
tasks for which at least one analyst chose a non-zero scale value. Thus, 
the scores do not reflect the cases of total agreement that a scale is 
not relevant to a task. This is therefore a more rigorous test than one 
including all the tasks, and an artificially high reliability or accu- 
racy score is avoided. 

Inter-task reliability would reflect the degree of agreement on scale 
values for overlap tasks. That is, since the same task can appear in 
the job of more than one performer, it can be scaled separately each 
time. This calculation can be made if sufficient task overlap data are 
accumulated. HSMS did not test for inter-task reliability. 



The skill scales have undergone several major revisions as a 
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'fhe .Standard Reliability Score can be compared from one situa- 
tion to another. The Standard Reliability Score for any analyst reflects 
his degree of agreement with the other analysts, in this case on a given 
scale, Ihe formula for the Standard Reliability Score is as follows: 

A = W , where: 

(N-1) (T) 

A^ = the Standard Reliability Score for an analyst, per scale; 

W = the relevant summed aKreement score (tiic sum of the num- 
ber of other analysts who have the same scale value for 
a given task on a given scale); 

N = the number of analysts involved in the test; 

T = the total number of tasks involved for the given scale, 

A^ is taken to two decimal places. Perfect inter-rater relia- 
bility for an analyst would be 1,00, 

Ac curacy 

Our meas ure of accu racy was actually a mejisuTe_of_C()iTigT 
the_gXQyP_ 's judgment . We assu.ned that the scale values arrived at through 
discussion in the Skill Scaling Conference were more likely to be correct 
than any one analyst's, especially when the analysts worked independently. 
Using the group's 5:cale values as the norm, we calculated the overall ac- 
curacy for each scale and the analysts* individual scores. The resulting 
measure is a form of reliability witu respect to the group norm. 

Since the accuracy measure is determined by conformity to a 
group judgment, a close relationship between this measure and the 
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reliability measure could result. It is possible, however, to have 
high prior inter-rater agreement and then have the team members reverse 
their independently arrived~at judgments during the Skill Scaling Con- 
ference. In such cases the accuracy measure could be lower than the 
reliability measure. The most desirable result is when the accuracy 
measures are higher than the reliability measures for each analyst, 
since this indicates that the divergence among analysts is in the di- 
rection of the norm. 

Standard Accuracy Score compares the individual analyst's 
scale values with the group answers. The formula for the Standard Ac- 
curacy Score is as follows: 

A = , where: 
T 

^c " Standard Accuracy Score for an analyst, per scale; 

C = the number of relevant correct answers (as given by 
criterion) ; 

T = the total number of tasks involved for the given scale. 

A^ is taken to two decimal places. Perfect accuracy for an 
analyst would be 1.00. 

TEST RESULTS 

Table 3 presents the suranary data for the HSMS pilot test. 
The scales are listed on the left in numerical order. The next column 
lists the number of tasks used as a data base (those of the 46 for which 
at least on'* analyst had a non-zero value for the scale). The Standard 
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fable I^. SUMMARY OF RELIABILITY AND ACCURACY DATA FOR SKILL SCALES 



Brief 
Name 

of Scale 




Analyst Name and Number 


Analyst Name and Number 




S 


F 


L 


M 


T 


I 




Aver- 


S 


F 


L 


M 


T 


I 




Aver- 




1 


2 


3 


4 


5 


6 


7 


age 


1 


2 


3 


4 


c 

J 


o 


7 


ace 


"j>a 


St 


:andard Reliability Scor^ 






Standard Act; 


;uracy 


Score^ 




1. Fre- 
Quencv 


46 


.80 


.81 


. 76 


. 78 


.68 


. 79 




•77- 


93 


.89 


.85 


.89 


.78 


.87 




. 87 


2. Loco- 
motion 


4 


.50 


.50 


.70 


.40 


.70 


.60 




.57 


. 75 


.50 


1.00 


.50 


1.00 


.75 




* 1 <J 


3. Object 
Manln » 


45 


.44 


.41 


.48 


.40 


.35 


.42 




.42 


.49 


.29* 


.51 


. 38* 


.73 


.82 






4. Guiding 
S teernfi. 


7 


. 74 


. 74 


.69 


. 31 


. 37 


. 74 




.60 


LOO 


LOO 


.86 




.43 


1.00 




7Q 


5* Human 
Inter * 


4u 


.42 


.48 


.47 


.54 


.42 


.47 




.47 


.52 


.57 


.63 


.87 


.61 


.74 






6, Leader- 


31 


.36 


.26 


.23 


. 32 


.31 


. 30 




.30 


97 


. 16* 


.26 




.23* 


.84 






?• Oral 
Lan^. 


46 


. ' ? 


.78 


. 78 


.04 


. 74 


.78 




.65 


LOO 


LOO 


L. 00 


.02* 


.89 


1.00 






8. Reading 
Lan£. 


46 


.79 


.70 


.80 


.75 


.52 


. 78 




.72 


.93 


. 74 


.96 


.85 


.57 


.93 




8'^ 


9. Written 
Lane • 


46 


.91 


.80 


.91 


.85 


.86 


.90 




.87 


100 


80 


Q8 


8Q 


.89 


.98 




Q9 


10 Meth- 
ods 


46 


.47 


.27 


.42 


. 37 


. 43 


. ^ u 




.41 


.67 


.30 


.54 


.72 


.67 


.78 




.61 


11 Qual- 
ity 


46 


.61 


.62 


.57 


. 35 


.40 


.52 




.51 


. 87 


.91 


.12 


. 41 


.46 


.78 




6Q 


12 Fig- 
ural 


45 


.48 


.16 


. 50 


.47 


. 45 


. 48 




.42 


8Q 


.20 




SI 


.47 


.93 




S<^ 


13 Sym- 
bolic 


38 


.56 


.26 


.64 


.63 


.62 


.62 




.55 


.66 


.34 


.82 


.79 


.74 


.92 




.71 


14 Taxon- 
omic 


45 


.55 


.15 


t48 


.S2 


.SS 


.52 




..46 


. 78 


.20 


.64 


.69 


.80 


.7§ 




.65 


15 Impli- 
cative 


46 


.25 


.33 


.27 


.29 


.25 


.30 




.28 


.50 


.54 


.39 


.50 


.46 


.9 




.30 


16 Finan. 
Err •Con, 


Hi. 


.71 


.60 


.69 


.49 


.60 


.68 




.63 


.88 


.81 


.81 


.52 


.64 


.93 




.76 


17 Human 
^ Err. Con. 
Col.Averag< 


46 


.61 
.59 


.64 
.50 


.54 
.58 


.55 
.47 


.60 
.52 


.61 
.59 




.59 


.70 
.80 


.83 
.60 


.63 
.71 


.61 
.56 


.63 
.65 


.89 
.85 




.71 
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^ Total number of tasks for each scale. ^ Line average of analysts' scores. *Accuracy less than reliability. 



Reliability Scores for each of the six analysts follow, and then the 
average across analysts* The Standard Accuracy Scores are presented 
the same way, on the right-h^nd bide of the table* The analysts' over- 
all averages are listed at the bottom* 

Ten scales showed acceptable levels of reliability (,50 or 
higher)* Revisions were then made for those scales requiring improve- 
ment* Given the fact of limited training time and the need to unlearn 
older versions of some of the scales, and given the tendency for reli- 
ability to increase over time, the results suggest that, with some revi- 
sions (made later), future results would be acceptable* 

KNOWLEDGE 

Generally, the "reliability*" tested and measured by HSMS re- 
fers to the extent to which several analysts in a team, exposea to the 
same performer at the same time, will independently arrive at the same 
data results* This is a measure of inter-rater reliability * In the 
case of knowledge identification HSMS adopted this measure for prelimi- 
nary testing* However, there are two limits to such a reliability test 
to bear in mind: 

1* It is impossible to measure the extent to which ana- 
lysts will be affected by other analysts' questioning 
of the performer about knowledge categories* There- 
fore, the independence of the raters is not totally 
assured* 

2* The HSMS test of knowledge category identification 
applies only to those subjects in the Knowledge Sys- 
tem which are relevant to the tasks covered in the 
test* Therefore, the reliability figures describe 
reliability for only a part of the Knowledge System* 
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The test for validity was translated to an accuracy mea- 
sure, i.e., a measure of conpruity with_jhe^^rwD Judj;^^ as was 
the case with skill scaling. We assumed that data arrived at through 
discussion in the team conference are more likely to be correct than 
any single analyser's, especially when the analysts work independently. 

Knowledge Identification Reliability and Accuracy Tests 

Because of time pres:7ures, the tasks of only one performer 
were involved in the testing. Ten Nurse Practitioner's tasks were se- 
lected. The tasks cover a range of the performer's activities and a 
range of knowledge requirements. C th- cen tasks, three required over 
fifty knowledge categories (as determined in the team conference) and 
seven required fewer than seven categories. 

Standard Reliability Scores were calculated for each taJk 
separately and for each analyst. Averages were then calculated across 
tasks and across analysts. In the case of knowledge identification, 
the Standard Reliability is as follows: 



Aj. = W , where: 

(N-1)(K) 

A^ = the Standard Reliability Score for an analyst per task; 

W = the relevant summed agreement sco'-e (the sum of the num- 
ber of other analysts who identified (or did not identify) 
all of the categories identified by any of the analysts 
for a given task) ; 

N = the number of analysts involved in the test; 

K = the total number of separate knowledge categories iden- 
tified by any and all of the analysts, per task. 



is taken to two decimal places • Perfect reliability for 
an analyst would be 1,00* 

Standard Accuracy Scores were calculated for each task sepa- 
rately and for each analyst. Averages across tasks and across analysts 
were also calculated. 

The formula for the Standard Accuracy Score in cases where 
each analyst may have identified a different number of knowledge cate- 
gories per task is as follows: 

= C , where: 
K 

A^ = the Standard Accuracy Score for an analyst, per task; 

C = the analyst's total correctly identified categories (as 
determined by the team conference), minus the total num- 
ber of categories identified by the analyst in excess 
of the correct total. (If the result is e negative num- 
ber, it is entered as a zero.); 

K = the total number of correct knowledge categories (as 
per the criterion). 

A^ is taken to two decimal places. Perfect accuracy for an 
analyst would be 1.00. 

Results 

Table 4 presents the results of the testing. The tasks are 
listed on the top row in order of the total number of categories iden- 
tified by the team for the task, from left to right. The analysts are 
listed in the far column. The upper portion of the table refers to re- 
liability, and the lower portion refers to accuracy. 
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Table 4. SUMMARY OF PILOT TEST RLLIABILITY AND ACCURACY 
DATA FOR KNOWLEDGE IDLNTIFICATION 



Analysts 






Tasks by 


Total 


No. 


of Correct 


Cate.-'or ies 


— 


6] 


52 


51 


9 


5 


5 


5 


Aver 
age^ 


3 


i 3 


2 


Aver- 
age'^ 




Standard Reliability S 


cores 






(S) 1 


. 66 


.62 


I .65 


.57 


.77 


.73 


.60 


.66 


r 

.60 


.77 


.69 


.67 


(F) 2 


.61 


.61 


.55 


.54 


.43 


.55 


.60 


.56 


.52 


.71 


.47 


.56 


(L) 3 


. 55 


.61 


.54 


.50 


.64 


.65 


.70 


.60 


.68 


.74 


.60 


.62 


(M) 4 


.66 


.51 


.59 


.66 


.77 


.49 


.70 




.63 


.28 


.25 


.69 


.56 


(T) 5 


.52 


.64 


.61 


.63 


.71 


.69 


.70 


.64 


.68 


.77 


.60 


.66 


(I) 6 


.58 


.60 


.66 


.68 


.81 


.55 


.60 


.64 


.68 


.71 


.67 


.65 


Average 


.60 


.60 

1 


.60 


.60 


.69 


.61 


.65 


.62 


.57 


.66 


.62 


.62 


Analysts 




Standard Accuracy Scores 




(S) I 


.72 


.75 


.74 


.33* 


.80 


.80 


.80 


.70 


.67 


1.00 


.50* 


.71 


(F) 2 


.61 




.55 


.44* 


.00* 




. 40* 


.42* 


.33* 


.67* 


.00* 


.40* 


(L) 3 


.52* 


.48* 


.47* 


.56 


.60* 


.60* 


.60* 


.55* 


1.00 


.67* 


.00* 


.55* 


(M) 4 


.59* 


-1 

.42* ' 


.33* 


.56* 


.80 


.60 


. 60* 


.56* 


.00* 


.00* 


.50* 


.44* 


(T) 5 


.43* 


1 

.44* 


.63 


.44* 


.60* 


.60* 


.60* 


.53* 


.67* 


.67* 


.00* 


.51* 


(I) 6 


.75 


.73 


.92 


1.00 


1.00 


.60 


.80 


.83 


1.00 


.33* 


1.00 


.81 


Average 


.60 


.59* 


.61 


.55* 


.63* 


.56* 


.63* 


.60 


.61 


.56* 


.33* 


.57* 



* Accuracy lower than reliability. 
^ Average of 7 tasks. 
^ Average of 10 tasks. 
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The data averages are shown for seven tasks and for ten 
tasks because the tasks with as few as two or three categories show 
somewhat distorted results for Analyst 4, The overall reliability aver- 
ages are not greatly changed, but the accuracy average is somewhat 
higher using seven tasks. 

The results indicate generally high average reliability by 
analyst, with none lower than .56, and an overall average of .62. On 
the other hand, the accuracy data indicate some problems in either over- 
or under-identif ication with at least one or two analysts. The overall 
accuracy is somewhat below the reliability average (.60 for seven 
tasks and .57 for ten). This reflects the deviation from the group 
norm for Analyst 2 in particular, and Analyst 4 in the ten-task aver- 
age. Analysts 1 and 6 show high accuracy. Analyst 6 was the trainer 
in this case, and influenced the team decisions in the direction of 
the content of the method. Analyst 1, the field team leader, showed 
high agreement with the correct identifications. 

Since this was the first test of the method, the results 
are encouraging; however, it is clear that team agreements, careful 
training, and review are necessary components of reliability. 

Knry^ledge Scale Relj^ability a nd Accuracy 

For testing reliability in the use of the knowledge scale, 
the units of observation are the knowledge categories rather than the 
tasks. A small number of Nurse Practitioner tasks provided over 100 
observations for use in the test of the knowledge scale. However, the 
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full range of the scale could not be tested, since none of the categor- 
ies identified were scaled above 5.5 on the knowledge scale. 

The ''correct" knowledge category identifications for a given 
task became the framev/ork for testing the reliability of the scale. 
Unlike the case of knowledge category identification, a fixed number of 
observations were involved. We calculated inter-rater reliability and 
an accuracy measure based on agreement with the scale values determined 
at the team conference. 

The formula for the Standard Reliability Score is: 

Aj. = W , where : 

(N-1)(K) 

= the Standard Reliability Score for an analyst; 

W =» the relevant summed agreement score (the number of 
ot:her analysts choosing the same scale value, for 
each category) ; 

N = the number df analysts involved in the test; 

K = the total number of categories involved in the entire 
test . 

The Standard Reliability Score was calculated separately for 
each analyst, and an average was taken for the scale as a whole. 

The formula for the Standard Accuracy Score is: 

Ac = C , where: 
K 

A^ = the Standard Accuracy Score for an analyst; 



A-28 



C - the number of correct answers as determined by the 
team conference; 

K == the total number of categories involved for the test. 

The Standard Accuracy Score was calculated for each analyst 
separately, and an average was then taken for the scale as a whole. 

Table 5 reports the test results. The scale shows a rela- 
tively lower average reliability (,47) and a much higher average accu- 
racy. The reason is again due to analyst deviation. Without Analyst 4, 
overall reliability is .55, However, unlike the case with other low 
reliability scales, the analysts (other than Analyst 4) were never more 
than one scale point away from the value arrived at in the team confer- 
ence. Since the reliability and accuracy measures treat any difference 
as equally wrong and magnify the e.rors, the results are more accept- 
able than may first appear. The lesson again is that training, team 
agreement, and review must be part of the method. 



Table 5, 



SUMMARY OF PILOT TEST RELIABILITY 
AND ACCURACY FOR KNOWLEDGE SCALE 



r 

1 




Analyst 


Name and 


Number 






S 


F 


L 


M 


T 


I 


Aver- 


1 Statistical Measure 


1 


2 


3 


4 


5 


6 




j Standard Reliability Score 


.49 


.53 


46 


.38 


.45 


.52 




Standard Accuracy Score 


.60 


.73 


64 


.38 


.54 


.88 


.63 



K " 113 separate knowledge categories 
N = 6 analysts 
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APPENDIX B 
THE HSMS **EDIT" PROGRAM 



INTRODUCTION 

The HSMS **EDIT** program was designed by Stephan Jasik and 
modified by Edward Friedman and George Chaikin to prepare the HSMS 
task data for use with the HSMS factor analysis programs PCVARIM and 
X2M0FA (Two-Mode Factor Analysis, Part One). 

The HSMS method calls for an unforeseeable number of key- 
punched data cards for the given set of tasks being studied. The num- 
ber of cards cannot be predicted because the number of knowledge cate- 
gories needed for tasks varies, and the total number of knowledge cate- 
gories identified for an entire set of tasks cannot be known ahead of 
time. Since it is impossible to predict the identity and number of 
knowledge categories, HSMS selected a data card format to handle re- 
quired knowledge categories as they are identified for the task, with- 
out knowing which are needed for other tasks. The format is not pre- 
determined in terms of preselected knowledge categories. Such a format 
could require thousands of cards, most of which voulo be^blank to indi- 
cate zero scale requirements for categories. Instead, the EDIT program 
creates a matrix of tasks by skill and knowledge categories; it enters 
zeroes in this matrix whenever a category is not required by a task. 

A data unit consists of all the cards for a given task, 

for which the ID name is the task's Code Number. For any given task, 

the task's Code Number appears on each card. Card TOO includes in- 
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formation on how many cards will follow for the given tasV and an 
abbreviated name of the task. Card TOl is in a fixed format, where 
fields refer to skill scales; it presents the task's scale values for 
each of the 16 skill scales. Any other cards are for knowledge cate- 
gories. These are numbered from T02 to TOn, and are set up so that 
each 8-digit number of the knowledge categories identified for the 
particular task is punched, each followed by its scale value. 

The EDIT program provides the user with information on the 
number, identity, and frequency of the skills and knowledge categories; 
it orders the categories, provides checks on the data, and makes pos- 
sible selection and/or logarithmic transformation of the data for sta- 
tistical use. EDIT therefore interfaces with PCVARIM and X2M0FA (Appen- 
dixes C and D) , and is always run when they are run. 

EDIT also provides dictionaries for the tasks and skill and 
knowledge categories in which the tasks* names appear in abbreviated 
form, and tasks and variables are shown with their internal numbers 
and Code Numbers. The other HSMS programs (except tor MATRIX) refer 
to the data by their internal numbers. 

DESCRIPTION 

EDIT is written in FORTRAN IV and was used in the Control 

Data Corporation's (CDC) 6600 computer at the Courant Institute of 

Mathematical Sciences of New York University. The operating systems 

in use during HSMS analyses were KRONOS and NOS (Network Operating 

System). EDIT was stored on magnetic tape in compiled and loaded form 

(i.e., in binary object code in non-relocatable form) and in OLDPL form. 
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The HSMS task data are transferred to magnetic tape in the 
form of an OLDPL, The data were maintained in numerical Task Code or- 
der in the OLDPL, A utility program, UPDATE, is used to transfer data 
"decks** to tape, correct data, or to generate temporary local files in 
the form of a compile file, which is part of the input file for EDIT, 
The input file for UPDATE is magnetic tape in OLDPL form, and the out- 
put file is C=DATA* The input file for EDIT is DATA, and the output 
file is TAPE9 (with optional printed output). 

The EDIT program is designed to receive the data decks in 
any order, but always with the cards for a given task in numerical or - 
der by card number , beginning with Card TOO, EDIT then performs the 
functions described below. 

E rror Checks and Listing of Data 

EDIT provides a variety of error checks and the option of 
listing or not listing the data cards as they appear in the input file. 
The listing is a default setting, and a NOLIST card is required to sup- 
press it. Some of the checks are suppressed by selection of NOLIST; 
others are carried out regardless of the option selected. The error 
checks and whether they are suppressed with NOLIST are shown below, 
HSMS uses NOLIST when EDIT is used with PCVARIM and X2M0FA, 

1, A check that the number of cards for the task is 
consistent with the number indicated on card TOO, 
Suppressed by NOLIST, 

2, A check that all the cards for a task have the same 
Task Code Number, Suppressed by NOLIST, 
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3. A check that data cards appear in proper sequence 

(as indicated in columns 1-3). Suppressed by NOLIST. 



A. A check that each task (Task Code Niimber) appears 
only once in the set of data. Suppressed by NOLIST 

5. A check that all punches occur in permissible col- 
umns; error messages indicate that either a blank 
or a punch should appear in a given column. This 
check is always made. 

6. A check that skill scale values all end in 0 or 5. 
This check is alwavs made. 

7. A check that all knowledge categories have a sc^ie 
value above 00. Suppressed by NOLIST. 

8. A check that knowledge categories appear only once 



for a given task. This check is always made. 
Options to Reduce the Number of Variables 

DELETE and CUTOFF are two options which make it possible 
to reduce the number of skill and knowledge variables which are 
copied to TAPE9. These options are used wl ^n EDIT interfaces with 
PCVARIM and X2M0FA (which are dimensioned for no more than 145 vari- 
ables) . DELETE is used to name specific knowledge and skill cate- 
gories to be eliminated; CUTOFF is used to specify the frequency at 
or below which skill and knowledge categories are automatically 
eliminated. 

Ordered Listings of the Data 

EDIT provides successive rearrangements of the data base 
(which exists as a matrix in which the skill and knowledge categories 
are the columns and the tasks are the rows) , and lists the resulting 
information in various formats. These arrangements are designed to 
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help the user select a frequency for CUTOFF, categories for DKLETK, 



and listings of the data in orders and forns useful for work wich 
other programs. The daca are structured as follows: 



1. Each skill and knowledge-* category is given an in- 
ternal number and is li: ted in the order in which 
it apPGars in the input file, together with its 
freqaency, anu the Task Code Number and scale value 
for each task in which it is scaled above 00. 
(This listing could be eliminated were the EDIT 
program to be revised.) 

2. The skill and knowledge categories are listed in 
descending order of their frequency of occurrence 
in the tasks. The original internal numbers, che 
Task Code Numbers, and the scale values appear 



3. The skill and knowledge categories are listed with 
the skills first (in a preset order) followed by 
the knowledge categories in ascending numerical 
order of their 8-digit code numbers. The original 
internal numbers, freruency, the Task Code Num- 
bers and the scale values appear agcin. (This 
listing could be reduced to just the identifica- 
tion numbers and frequencies we; <5 the program to 
be revised . ) 

The variables copiod to TAPE9 are renumbered in- 
ternally and listed in order with the skills first, 
followed by the knowledge categories in ascending 
order of their 8-digit code numbers. Frequency 
data are listed, but no task and scale value data. 
If DELETE and CUTOFF options have been selected 
this listing does not include the eliminated cate- 
gories. (In the previous three listings categories 
to be eliminated as a result of DELETE or CUTOFF 
options are indicated by asterisks.) 

5. EDIT provides a "Task Description Dictionary" which 
lists £'ch task's observation number (its internal 
number in numerical order) and its actual Task Code 
Number, tcgeth er with the abbreviated name of the 
task as found in card TOO for the task. 

6. The final numbers assigned to skill and knowledge 
categories and to tasks are the numerical referen- 
ces ^or observations (tasks) and variables (skill 
and knowledge categories) in PCVARIM, XIHOFP y and 
X2MFA2 . 



again . 
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Logarithmic Transformation of Data 

I^Then EDIT is used to interface with PCVARIM or X2M0FA, the 
NORMALIZE option permits a logarithmic transformation of the data to 
adjust for a large number of zeroes in the data base, (Tasks which 
do not require knowledge categories required by any other task are 
scaled at zero,) The NOR^IALIZE option follows the formula: 

X = SQFT (X + 0,5), where X is a scale value. 

COMMENTS 

EDIT places heavy demands on a computer's core memory. 
The size of the matrix array of data (36000 cells) is such that it 
v,;as necessary to compress the data so that 3 units are stored in a 
word. In its present form EDIT can be used only with a computer that 
has a 60-bit word, EDIT requires 200K of core memory to run, and is 
dimensioned to handle up to 700 tasks, assuming a task-by-category 
matrix of 36000 cells. Because there is a wide range in the number 
and frequencies of variables that can be associated with a set of 
tasks, it is virtually impossible to estimate beforehand the precise 
number of tasks which can actually be handled in a particular rui,: HSMS 

has successfully run EDIT with as many as 560 tasks. With a data base 
larger than this there may be a risk that a time limit may be reached 

before all of the functions of the program have been carried out, 

even though the printed output will be complete. 

The EDIT program was designed early in the history of HSMS, 

and, therefore, includes procedures and options which we now see can 
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be eliminated or which should be able to be suppressed* For example, 
once new data are checked and variables have been selected for use in 
PCVARIM or X2M0FA , the EDIT checks, restructurings, and most listings 
are unnecessary. The reader is encouraged to revise EDIT to make it 
more efficient. HSMS would have done so were it to have continued its 
operations . 

SYSTEM SUBROUTINES CALLED BY EDIT 

The EDIT program which is presented in this appendix calls 
for a series of subroutines. Among these, EXIT, TIME, and DATA have 
not been included in the listing, since they are systems-' ased. It is 
assumed that the user can utilize comparable routines after reading 
the descriptions which follow."^ 

EXIT 

This subroutine terminates program execution and returns 
control to the operating system* A STOP statement may be preferable. 

TIME(a) 

This subroutine can be used as a function or subroutine. 
The value is returned via the argument and the normal function return. 
The subroutine returns the current reading of the system clock as the 
value of the argument a or of the funcf^.on in the form lOHbhh.mm.ss .b, 
where b denotes a blank, and hh, ram, and ss are the number of hours, 
minutes, and seconds, respectively. The value returned is Hollerith 

1 

INPAGE is called and does appear in EDIT. It is an entry point ap- 
pearing in PAGER. 
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data and can be output using an A format specification. The type of 
this format is real. 

DATE (a) 

This subroutine can be used as a function or subroutine. 
See CALL TIME(a), above. The current date is returned as the value 
of argument a or of the function in the form lOHbmm/dd/yyb (unless it 
is changed at installation option), where b denotes a blank, mm is 
the number of the month, dd is the number of the day within the month, 
and yy is the year. The value returned is Hollerith data and can be 
output using an A format specification. The type of this function is 
real. 
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STRUCTURING THE INPUT FILE; EDIT WH.^ USED ALONE 



In the set-up presented below, the NOS system in use in 1977 
at the Courant Institute with the CDC^6600 is assumed. It is also as- 
sumed that the reader can refer to the UPDATE Reference Manual or a 
counterpart program. In the following instructions b denotes a blank. 



INPUT FILE FOR EDIT USED ALONE 



Name 


Keypunch Cards (one per box) 


TnQt"T"iir»t"'f r»Ti c 


Sys- 
tems 
Cards 


,EDIT ALONE 


Identification niirnHpf ;5nH 
0 ther information depending 
on svstem. 




USER( ) 


USER and user code. 




CHARGE ( ) 


Charge card. 




HEADER, EDIT ALONE 


Optional to print out head''' ig 
across a whole page. 




LABEL {OLDPL,VSN=T ) 


Identifies OLDPL tape. 




UPDATE (Q,D, 8, C=DATA) 
or 


When selecting tasks using 
*COMPILE form of UPDATE. 




UPDATE (F,D, 8, C=DATA) 


When using all data in OLDPL, 




UNLOAD (OLDPL) 


Unless being used for more 
than one run per submission. 




LABEL (PROGRMS , VSN=T ) 


Identifies program tape . 




C0PYB7?(PR0GRMS , EDIT) 


Copies progr.im to local file. 




SETTL(IOOO) 


Time limit. 




RFL (200000) 


Defines field length. 




EDIT. 






End of Record Card 




In- 
put 
Cards 
for 
UP 

DATE 


*COMPILEb , , 
etc, , or 

*COMPILEb , 
(option) 


Calls tasks from OLDPL, Each 
card starts: *COMPILE in Cols, 
1-8; Column 9 blank, followed 
by Task Code Numbers separated 
by commas; last entry has no 
comma. Tasks can be in any or- 
der, 'Jonsecutive Code Numbers 
can be called as last entry on 
cavd after comma or as only 
entry on card by writing first 
Code Number, then a period, 
then last Code Number of the 
series, A new card must fol- 
low. 




End of Record Card 


Needed even if no UPDATE cards 
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INPUT FILE FOR EDIT USED ALONE (continued) 



Name j Keypunch Cards (one per box) 



In- 
put 
Cards 
for 
EDIT 



TITLEb b b b b etc 



NOLIST 

(option) 



DELETEb b b b _ 
(option) 



CUTOFFb b b b 
(option) 



Instructions 



Columns 1-5: punch TITLE; 
Columns 11-60: punch title of 
program, date, and any other 
special identification of 
this run. 



Only if listing of data and 
some error checks are to be 
suppressed. 



One card for each individual 
skill or knowledge category 
to be deleted from input data 
(if not to be deleted oy 
CUTOFF) . 

Columns 1-6: p'mch DELETE; 
Columns 11-18: punch 8-char- 
acter skill code or 8-digit 
knowledge category code num- 
ber. 



To eliminate skill or knowl- 
edge^ categories at or below 
a selected frequency from in- 
put data. 

Columns 1-6: punch CUTOFF; 
Columns 11-12 : punch selec~ 
ted frequency (usually 4 or 
more) left justified in 
field. 



NORMALIZE 

(option ) 



End of File Card 



Not usually used when EDIT 
is used alone. 



B-10 



ERLC 
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OPT-1 

IT 



FT^« 4.6*428 



77/09/15 



OVERLAY( EDIT»0>0) 

PROGRAM EDITFA( INHUT* OUIP JT» D AT A, T APE9, 1 APE 1 «D ATA, TAPE 5« INPUT, 
1 TAPE6-JUTPUT) 

CaHMON/TA8LES/I60T,ITTAE(700),KC(700 ), K FREw ( 700 ) , KPT ( 700 ), I W( 700 ) , 
1 NXT,HTAB( 36000 ),NT AS K,>'A<TAB,hAXKC 

C0MH0N/CaLS/IT00(3,2),IT0N(2,2), I TO 1 ( 16, 2 ) , ITO 2 ; d, 2) 

COMMON/ 10/ INPUT, IQUT, I TAPE, IC ARD (80 ) , L INCnT, L I STPR ( 1400 ) 

COMMON/SKI LLS/KEYWD( 16), KEYAL 8 (16) 

COMMON/DEL I ST/NDEL, LD EL ( 700 ) # KU TQFF 

COMM0N/L06/TTl,LN0RM,LPPINT 

CQMMON/TCARD/ITDCN,ITIN,NC 



LOGICAL TTl,LNaRM,LPRINT 
LOGICAL SEARCH, Li, NEWTSK 



OlrtENSION KCREA0(8) 
DIMENSION KCBUFF(700), ISKV(700) 
DIMENSION LIST(3),LISTR(2) 
DIMENSION PRLIS1(700) 
CIMENSION IFMAT(3) 

EOUIVALENCE (LIST(l), ITDCN) , ( L ISTPR (1 ) , PR L IS T( 1 ) ) 
DATA KXT/1/, IBOT/0/,«AXTAB/ 36000/, HAXKC/700/ 
DATA TTl/. TRUE. /,LNORM/. FALSE./, LPR In T / . TRUE . / 

ERIC 



E3ITFA 

EDITFA 

EDITFA 

EDCQH 

E'^RG 

ENLRG 

EDCQN 

EDCL'H 

EOCOI 

E^LRG 

EDCDf< 

EOCOM 

EDCOrt 

E^LRG 

EOCCH 

EJCOM 

EDCOM 

EDCDM 

EDCOf^ 

E0CC11 

EOCCM 

EOCDM 

EDCDM 

EOCCH 

EDCDM 

EMLRG 

EDCOM 

ENLRG 

EDCOM 

EDCOM 

EDCOM 

EDCCM 

ENLRG 

INTDTA 

ISTDTA 
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DATA KEYl«0/8RL0C0M0T'<#8RuBJ f*ANP,8KGDG-ST.<&, 8KhUM iNTk, BKLf AO jHl P, 
28R0RAL uSe#8PREAD USE,8RW^IT USE , SRMETHCOi ,6K0UALnY ,8KFIGu»^AL , 
38RSYM80LIC»8RTAX0N0HC#8RI»1PLICIT,6RFInC EKK,8kHU««J ERR/ 

DATA KEYAL3/l#2,3,<r,5#6#7#8#9,10, 11, 12, 13 # 1^, 15 # 16/ 



DATA 

* 

DATA 



(IT0O(n>I«l,3)/2*5*29/* 
(IT0OCn*I-^f 6)/3*10*30/ 
rXT0NCI)*I-l*2)/2*5/* 
(IT0NCI)*I-3-^)/3* 10/ 
(ITOICI)*!- 1*16) 
35#38#^1,4«#<»7# 50; 
(ir01CI)*I-17*32) 



DATA (ITOICI)*!- 1*16) 

I /32#35#38#^1*4«#<»7#50#53»56#59#62/65, 68»71»74#77/# 
(ir01CI)*I-17,32) 

78/ 



X (ITOICI), 1-17,32) 

2 /33»36#39»42»45*^8*51,5^,r57,60, 63#66*69#72» 75# 7 
DATA (IT02(I)*I- 1, 8) /30,39,^3#52,56#65,69,7b/, 
2 (IT02(I)*I«9«16) /37M0,50#53#63#66»76#79/ 



INPUT - 5 
ZOUT > 6 
ITAPE ■ 9 

CALL 0VERLAY(*HEDIT»1#0) 
INPI'T • 1 
REWIND ITAPE 

CALL 0VERIAY(*HEDIT#2#0> 
REWIND ITAPE 

CALL 0VERLAY(*HEDIT,3,0) 
REWIND ITAPE 
CALL EXIT 
END 



INTDTA 

INTDTA 

IMDTA 

i\TDTA 

I.\TDTA 

iNrOTA 

rUDTA 

INiTDTA 

iNTDTA 

INTDTA 

IMTDTA 

nTDTA" 

Ii^TDTA 

INTDTA 

INTDTA 

INTDTA 

INTDTA 

iNTDTA 

INTDTA 

HTDTA 

INTDTA 

INTDTA 

INTDTA 

EDIIFA 

EDITFA 

EOITFA 

EDITFA 

EOITFA 

EOITFA 

EDITFA 

EOITFA 

EOITFA 

EOITFA 

EOITFA 

EOITFA 



ERIC 



SUBROUTINE PAGERCI) 
COMNOh/SKIP/LINSKP 

COHHCN/ 10/ INPUT, iaUT# ITAPE, IC ARD { 80 ) # L I SCNT# L ISTPR ( 1 '.OO ) 

COHMON/LQG/TT1#LNORM,LPRINT 

LOGICAL LPRINT 

DIMENSION I{1)#IHEAD( 12), ISUBHDdl) 
DATA J/0/,K/0/,NPAGE/0/,IHE AD/12+10H /, 
1ISUBHD/11*10H / 

C AT THIS ENTRY POINT THE PARAMETER I IS NOT USED 

10 NPAGE ■ NPAGt ♦ 1 

WRITE (I0UT#1) IHEAD#NP AGE 

1 F0RHAT{1H1,12A10,*PAGE*,U) 
fciRITE{I0UT#2) ISUBHD 

2 t^ORMATtlHO, IIAIO/ ) 
LINCNT • 2 
LINSKP • 0 
RETURN 

ENTRY INPAGE 
K • K ♦ 1 

IF (K.GT.l) GO TO 30 
DO 20 L - 1,7 
20 IHEAD(L) • I(L) 
CALL TIMEdH) 
lHEADf9) « IH 
CALL uATE<lH) 
IHEADdl) • IH 
ISUSHDd) • lOHCARD SO. 
■ISUB,ID{2} • lOHlNPUT DATA 
GO TO 10 



PAGEk 
PAGER 
PAGER 
EnLRG 
PAGER 
PAGER 
PAGER 
PAGER 
PAGER 
PAGER 
PAGER 
PAGER 
PAGER 
PAGER 
PAGER 
PAGER 
PAGER 
PAGER 
PAGER 
PAGER 
PAGER 
PAGER 
PAGER 
OAGER 
PAGER 
PAGER 
PAGER 
PAGER 
PAGER 
PAGER 
PAGER 
PAGER 



ERIC 
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r 



30 



IF ((H.63.2).Af;t.LKKlhT) jL 
IF (K.EQ.2) RtTLf<4 
L • K - 2 

60 TO (<|0>70>80>90>100>110)L 



Tj IC 



UUBHD( 
ISUBHD( 
ISUaHD( 
ISUdHU( 
ISUBHO( 
ISU8HD( 
ISUbHO( 
ISUBHDC 
ISUBHD( 
ISUBHUdO) 
I3UflHD( 11) 



1) 
2) 
3) 
<•) 
a) 

6) 
7) 
8) 
9) 



lOri lUT VAR 
lOH COE FR 
10HE5 

10HTA:)^S IN w 
lOHHICH VAftiA 
10H3LE AFJ>tA% 
lOhLD - f-RI 
lOHMEj AS wA 
ICHKIABLE ceo 
lOHtS APFEARE 
lOhu IH I.>JPuT 
+ 1 



50 NPAGE • NPAGE 
IX « 0 

WHITE ( I.)jr, 1) IHEAD,»*PAGF 

CO 60 L • 1»2G 
60 WRITE(IjUI.3) 
3 FCR'^ATdH ) 

WRITE ( IuvJT><i) ( I:.UJHD (L ) >L=«4 .li) 
^ FOR-Al (26)(> 8A10) 

i^KlTEd jJI,i) 

kKUEC l ju\,t>n 

5 FORlAK^Ojk,^** DlbllNCl VA^UbLt CCbi 
lTA»////32A>*ASTckIjA MA,<< tfARlA.Kc.S I 
2k GIiRhCriVE*) 
GC T.' 10 
70 I i('BHD( 7) « lOHfc J - i y<i 

lOHlEJ or 
10ri.oL..CY '.F 
lOHVAKlAtJLc : 



3E -J 



. L c T t J 



In Trt£ j1 da 
01 e T'. CUT Jf F r 



lit'BHDC 7) 
IS'JoHDC 3) 
ISj3Hu( 9) 
lSUdHj( 10) 
ISUoHD( 11) 
6 'J TO "30 
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»AGE< 
^AGER 
"AGFR 
PAGER 
♦»AGER 
.'AGER 
f AGER 
JAGEi? 
•»AGER 
3AGER 
•>AG£R 
>>AGEK 
i>AGbR 
"AGER 
'AGER 
PAGER 
•'AGER 
•'AGbP 
-AGER 
PAGER 
^AGbR 
•'AoER 
'AGER 

- AGt K 

■•AGf K 
f'ACbR 
•J AGtR 
PAGER 
■>AGt» 
■•AGf R 
J AoER 
PAGER 
PAGES 
-AGbR 
PAGER 



BO iSUBHOt B) 
ISUBHD( 9) 
ISOBHDi 10) 
ISU3HD( 11) 
GO TC 50 
90 ISUBHDi 7) 
ISUBHD( 8) 
ISUBHDi 9) 
ISUBHD(IO) 
ISUBHD(ll) 
GO TO 50 

100 ISUBHD( 1) 
ISUBHD( 2) 
ISUBHO( 3) 
ISUBHO( ^) 
ISUBH0( 5) 
ISUBHO( 6) 
ISUBHD( 7) 
ISUBHD( B) 
ISUBH0( 9) 
ISUBHO(IO) 
ISUBHOdl! 
60 TO 50 

110 ISUBHO( 1) 
iSUBHOi 2) 
iSUBHDi 3) 
ISUBHDi <») 
ISUBHDi 5) 
iSUBHDi 6) 
ISUBHD(7) • 
'iSUBHDtB) - 
ISUBHD(9) - 
ISUBHD(IO)- 
ISUBHD( ID- 
GO TO 10 
O END 



' lOHTED Br ASC 

' lOHENDIiG CRO 

' lOKER OF VARI 

> lOHABLE COOES 

' lOHED - DEL 
' lOHETED VAdA 
I lOHBLES - NOT 
lOH WRITTEN Q 
I lOHNTlJ TAPE9 

lOHVAiUABLE H 
lOHO. VAR CQ 

' lOHDE FREC 
lOH FIMAL CQ 

I lOHRRESPONDE.S 
lOHCE OF VARI 
lOHABLE NUMBE 
lOHRS TO VARI 
lOHABLE CODES 
lOK AS WRITTE 
lOHN ON TAPE9 

10H0BSERVATI3 
lOHN NO. TA 
lOHSK NO. 
lOHTASK DESCR 
lOHIPTION Die 
lOHTIONARY 

lOH 

lOH 

lOH 

lOH 

ICH 



15S 



PAGER 

PAGER 

PAGER 

OAGER 

PAGER 

PAGER 

PAGER 

PAGER 

PAGER 

PAGE 

PAGEk 

PAGER 

PAGER 

OAGER 

PAGER 

PAGER 

aAGER 

PAGER 

PAGER 

P<iGtR 

PAGER 

PAGER 

PAGER 

PAGER 

PAGER 

PAGER 

PAGER 

PAGER 

PAGER 

PAGER 

PAGER 

PAGER 

PAGER 

PAGER 

PAGER 

PAGER 



LOGICAL FJ^iCTIQ^ SEA.iCH( I rE.->, INO, L 1ST) 

DIMENSION LIST(INO) 

SEARCH « .TRUE. 

IF (ITErt.tQ.O) RETUR-^ 

IF (iNO.Ew.O) 61- TO 20 

00 10 I « l#IND 

IF (LiSHI) .NE.ITE.1) 6u iQ 10 

SEARCH ■ .FAL5E. 

RETURN 
10 CONTINUE 
20 IND ■ INO ♦ 1 

LISKINO) - ITE.M 

RETURN 

END 



CVERLAY(E0IT,1,0) 
PROGRAM ROOIR 

0I*1ENSIJN I0JR(6),iC(7),jC( iO) 

1 NXT#MTA6( 360CO) ♦NTA5^»^A<IAb,MAl(^C 
CCMf'nN/c jLS/ITOO(3,2), HO-K 2, 2 ) , I jqK lb, 2) , I ru,i { j, 2 ) 
CGI.M-JN/ 10/ Input, IL'UI, I TA,»£, ICAKD{ :50),LIf,C >ir,LiiT^R{UOO) 
CQW.h/'iKlLLS/Hii.Oi lo),-<EYAL3(16) 
CO*"« iN/DELI jTZ-vDEL, LJtL {7J0>,K T. FF 

Cu>'M':r./LQG/rri,L\^(<M,LPi<isT 

COn*.:NMCA^C/IIDC oITU»NC 159 



iEA-^CH 

SEARCH 

SEARCH 

:>E ARCH 

ScAPCH 

ScAKCH 

SEARCH 

SEARCH 

>b ARCH 

iEARCH 

ocARCH 

J^E ARCH 

iE ARCH 

SEARCH 

•OJIR 
•<OOIR 
-ODIR 
RJJIR 
E3C',M 
6SLRG 
EmLRG 
EDCuM 

£ JCOM 
t\LftG 
tDCC 
EDCCM 
E3CL'' 
t tL R G 

r:ocr.*» 

t jCC-^ 
cDC'.'M 



ERIC 



LOGICAL TTl,LNCjRM*LPRINl 
LOGICAL SEARCH*LS>NEWTSK 



DIHENSI3N KCREAD(8) 
DIMENSION KCBUFF(700) > ISKV(700) 
DIMENSION LIST(3),LISTR(2) 
DIMENSION PRLIST(700) 
DIMENSION IFMATO) 

EQUIVALENCE ( L IS T ( 1 ) * ITDCN) , ( L I S TPR ( 1 ) , PR L I i 1 ( 1 ) ) 

DATA IDIR/IOHTITLE tlOHDELETb tlOHCUfOHF > 10HNu^<^" AL I ZE 

1 lOHNOLlST *10H /* lAR/lOHttttttrttt/ 

DATA ISL/IOH / 



NOEL - 0 

KUTOFF ■ 0 

NUM • 0 
10 READ(INPU:*1)ID, IC 
1 FORMATtSAlv-)) 

IF(fcOF( INPUT) )20* 30 
20 RETURN 
30 NUM • NUM ♦ 1 

DO 40 I"l*6 

IF ( ID.EQ. IDIR( n ) GO TC 50 
40 CONTINUE 

LiNCNT ■ LINCNT ♦ 2 

IF (LINCNT. 6T. 55) CALL PAGER(56) 

tiRITE( IUUT#2)NUK#I0,IC 

kiRITE(lUUr>3) 



EOCOH 
EDCCM 
tOCO^I 
EOCCh 
EOCCM 
EOCQw 
EOCGM 
cNLRG 
60C0M 
ENLRG 
EDCGM 
EDCOM 
EDCOM 
EOCC* 
> KOJIR 
RDOIR 
ROOIR 
RDDIR 
RODIR 
•<0l}IR 
RDDIR 
RDDIR 
ROOIR 
RCDIR 
KODIR 
RDDIR 
RDDIR 
RDOiR 
RODIR 
RODIR 
RODIR 
RODIR 
RDDIR 

r^ODlR 



3 FuR^-AT', 7x,8{ lHt),« - iN.-^tCCSMZASLt Ol^LCllvb*) 
60 rO 10 

50 60 TC ( iOO>2GC, 300> <iOO, pO J# 1000) 1 
100 IF (TTl) G3 Tf 110 

LI^iCM » LINCNT ♦ 2 

IF (LlNCNF.GT. 55) CALL PAGEi<(56) 

• RITfc ( I0UI»2)NU>«, ID, IC 

•»RITF(I jUT,4) 

<• F!:RKAT(7X,8(lht),* - A llfLt HA.1 ALt<LAOY oEts «fcAD») 
GC TC 10 
110 CALL i.vPAGE(IC) 
TTl • .FALSE. 
GU TC 1000 
200 t»ECCOE( 1C,9» IC ) jC 
9 FQRMAl ( lOKl) 

IF(jC(i)-lRO) 210, 230,230 
210 DECr.DE(6,ll,IC)jC 
11 FGR'^AIlKd) 
Du 220 I«i,l6 

IF (jC.E;i.\dYV.D( I) ) 1.1 Z'iO 
220 COMlNuE 

LiNCr f » LISC^ T + 2 

IF ( L KCnF .bT.5^ ) CALL -A;fi'(t)6) 

•.R I Tc( ls)Ll ,2 )M ^ , iD, R 

fefi ITE (I.:ol,5) 
5 FCR*AT( 17x,otlHt luo-Z 

GJ T? 10 
230 kCDEL » 0 

235 I«l,c 

IF ( ( jC( I ) .LT. IPO) . .-^ . ( j: ( 1 » . ^1 . Ik^) ) T 
235 kCCEL = 10*«C0£L ♦ (Jo{I)-l-<0» 
I • kCOEL 



1 ZAjLh ?K ILL* ) 



■i'J'Jlfi 
-JOOIR 
R30IR 
RODIR 

RJJIR 
»^ JOIS 
K J J IR 
■<DuIR 
DDIR 
jJlR 
-^30 IR 
RODIR 
■i JulK 
■<DOIR 
RDOIR 
•<i)DIR 
■^DDIR 
R3DiR 
0 J I f< 
^DDIR 
RDi) I« 
XDDIR 

k:::,ip 
r:dr 

kDuIR 
Ik 

fOuI? 
-s JJ l*^ 
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ERIC 



240 LS - SEARCHi I,ND6L,LDEL) 
IF (LS) 60 TO 1000 
LINCNT ■ LINCNT ♦ 2 
IF (LINCNT. 6T. 55) CALL PAGER(56) 
WRITE( IQUT>2)NUH> ID, IC 
l»RITE( I0UT,6) 

6 F0RHAT(17X,8(lHt),* - THIS KC HAS ALREADY BEEN DELETED*) 
60 TO 10 

300 DECQDE(10,9,IC) JC 
I ■ 1 

IF (KtTOFF.EQ.O) GO TC 310 

LINCNT ■ LINCNT ♦ 2 

IF (LINCNT. GT. 55) CALL PAGER(56) 

hRITE(I0uT,2jNUr,ID, IC 

bRITE(lQUT,6) 

8 F0RMAT(7X,6(lht),* - A CUTCFF HAS ALREADY BEEN READ, THIi CARJ 
IL BE IGNORirO*) 
GO TC 10 
310 J " JC(I) - IRO 

IF (J) 320,350,330 
320 LINCNT ■ LINCNT ♦ 2 

IF (LINCNT. 6T. 55) CALL FASER(56) 
WRITE( I0Ul,2)NUKy ID, IC 

hRITE( I0UT,7)(IBL,N"1, I),IAR,(IBL,*1"I,8) 

7 F0RhAT(16X,9Rl,* - DIGIT iUST APPEAR HtRt*) 
GO TO 10 

330 lF(JC(I).Ea.iR ) GO TO 1000 

IF (J-9) 350,350,320 
350 KUTOFF ■ 10+KUTCFF ♦ J 

I ■ I+l 

GO TO 310 
^00 LNORH ■ .TRUE. 

60 TO 1000 



RDDIR 
RDDIR 
RDOIR 
RDDIR 
^DDIR 
KDDIR 
ROOIR 
RDDIR 
RDDIR 
RDDIR 
RDOIR 
ROOIR 
RDDIR 
RODIR 
RDDIR 
wIL RDOIR 
RDDIR 
RDDIR 
RDOIR 
RODIR 
KDDIR 
RDDIR 
RDDIR 
RDDIR 
RDDIR 
RDDIR 
RDDIR 
RDDIR 
RDDIR 

RDDIR 
RDDIR 
RDD'R 
•< JOiR 
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500 lPi?iNi( » .FAlSt. .ODIR 

1000 LI'^ICM = lI'^CnT + i •R3DIR 

IF iLIf4C-<7.GT.55) CALL Cft3E-((56) «^uIR 

*f<ITc ( lU:^rf 2^)0^,1 J, IC RDDIR 

2 FOR-^AH iJ(,I3»3X»dAiGI ■<OuIR 

GO Ta 10 RDDIR 

END kDOIR 



C;veRLAY{EDIT*2,0J 
PR06RAf» RDTAbK 

COMMuN/rADLti/IEcT, HTA6( 7OO),KC{ 700 )* <FR£ 700 ) , K ? T ( 700 ) , U ( 700 ) , 
I NXTtMFAfJ t 3oOOO) »NTASK,f AXTAB».MAXKC 

Cufr«JN/C JLi/IT00(3,2), ITOm(2,2),I TOK 15,2), iroZ (8,2) 
COr«iCN/Ij/I\^lT, lutr, I TAPE, I CARO( 30), LlNCNr»L U-T^R (UOO) 
CDIir-N/SKlLLS/fc Y^'H l6),<LYALo(lo) 
C !**-- ^/JtLl5T/\0LL, LjLL {7J0) ,M)T j>F 

c':*•''i•^/L,.o/^l,L^v'K % Lr'i^ I n 

C J-^^-JN/TCAkC/ IluCN, inN,--c 



L:GICAL TI1,L^ Jf.^,L"i^I NT 
LLGICal ShAKCH,Lb,M. ^T3k 



'OTAS'* 
RJfASK 
tOCC" 
t \ LPG 
EMkG 

i: JC Gf^ 

t vLRG 

c jCL^ 

EDO" I 

JCi - 
E jC '^"^ 

5 3cr 
e:c. " 
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ERIC 



DIHENSIJN KCREAD(d) 
DIMENSION KCBtFF ( 700.) * I SrV ( 700 ) 
DIMENSION LIST(3),LiSTR(2) 
DIMENSION PRLIST(700) 
DIMENSION IFMATO) 

EQUIVALENCE ( L IST( 1 ) * I TDCN) , ( L I S TP R( IJ *P« L 1ST ( 1 ) ) 



CALL INPA6£(NtK) 
NUM » 0 
NTASK ■ 0 

10 READ( INOJT/I) ICARD 

1 F0RMAT(80R1) 

IF (EOF(INPUT)) 20,30 
15 WRITE(I0UT,6) 

6 FCR«AT<20(2H •),♦ ERROR - PREMATJKE ENO UF FILE DN INPUT OATA*) 
20 RETURN 

30 NUM • NJM ♦ 1 

IF (.NOT.LPRINT) 6C TO 35 

LINCNT • LINCKT ♦ 2 

IF (LINCNT. 6T. 55) CALL pA3ER(56) 

WRITE(I0UT,2) 

2 FOR^ATdH ) 
bRITE(IUUr,3)NUf*,ICARD 

3 FORMATt 5X,I4,5X,80R1) 

35 CALL NU«BER( IT00,LIST,3) 

NE*/TSK ■ SEARCHt ITIN, NTASK, ITTAd) 

IF (KEhTSK) WRITE(ITAPE,7) iriN,(KARD(I)»I«31»b0i 

7 F0RMATtI5,50Rl) 

IF (.NOT.LPRINT) TG 'rO 

IF (ITDCn.NE.C) CALL PERRlRd) 



EDCOM 

bNLRG 

f^OCC*< 

ESLR6 

E3C0r- 

EDCGM 

EOCCM 

EOCOM 

kOTAS< 

ROTA^K 

RDTASK 

ROTASK 

ROTASK 

RDTASK 

><DTASK 

RDTASK 

RDTA5K 

rOTASK 

ROTASK 

R3TASK 

R3TASK 

RDTASK 

RDTASK 

ROTASK 

RDTASK 

RDTASK 

ROT^St^ 

ROTASK 

ROTASK 

RDTASK 

ROTASK 

RDTASK 

RDTASK 

RDTASK 



ERIC 
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100 

40 

1 

Call page<{56) 



IF (nC.LE.O) call 
IF (NC.LE.OI NC » 
IF iKt^iSK} GC Ij 
LINCM « LINCNT ♦ 
IF <Ll>ICNr.GT.55) 
>RITt{ I'jur,9) 
9 f£l«»'AT{10{2H ♦),♦ THii TA>« 
ICCJRENCfc *ILL bE IG^lCREC*) 
READ{ I.>»PJT, DICARO 
IF {E!jF{ INPUT) ) 13>t>0 

IF { .N jT.LJ'-iiNT) G'j Tj 80 
LUCNT • LINCM ♦ 1 
IF (LInCnT.GT.55) call FA3hK{3fc) 
WKITE{ IJuT»3)NtM»ICAKa 
CALL ^J«3fcR{ ncN,LISTft,2) 
60 IF (LIbf■<{l).^E.{LIi>T(l) + l)) CALL r=K^,ft{l) 
IF UISW{2).NE. ITI\) CALL r>t.-<R^iR ( i ) 
IF (LliTRd )..vc.O) GJ TC iO 
LI-'CM • LInCjT + 1 
IF ( LInCnT.G1.;>5. ) CALL ~Ajc>{:>6) 
•►^ITECI juT,3) 
3 f!>R-*Al(» Adi-VL Ca^u APPtA^i 7 J 
I RE''Kl>,rEj sELiw AS IF IT "l^z 



HAS ALkEAOY Af»;icA^hu I» IncjI - THIS 



.TA-^T 



■J I- « 



GC Ij 30 
(50 IND « lo 

CALL NC-'dt>«(iroi,IiKV,Ic» 
OC 52 I«lfI6 

IF ("i J(IiKV{I)»3).-.:».0) 3l 
LINCM » LInCM f I 
:f (LlMC-n.GT.:>::) CALL CAGt-<(j 
--^I ltd .U'lf II )I 



rAS" - n •III le 



52 



kDI ASK 

.^3rAS« 

!<J TA?K 

K'JTAS'' 
so TASK 

<<DrAS-< 
RDl ASK 
•OTa3< 
KCT ASK 
RDTASK 
•^DTA.NK 
^OTAS- 

:<3fASK 

■< jFaSk 

^3TA^^ 
-{DTaS" 
RDF ASK 
K D I ASk 
•OIASh 
-OTAS* 
■<DTAb< 
•<'jrA5i 

■< .1 T A 5 K 
kDTAS;^. 
•<3T A'Ik 
■<D I AjH 
^J(AS-< 
-'OTASh 
J T A b '\ 



jn5 



^M^slf^** ^'^^^^ ' ' ^^^^-^ "^'-'^^ NO.*, 12,* JOES NOT END WITH A 0 OR 
82 CONTINUE 

IF CNEWTSK) CALL PUTL 1ST ( IND, KE YAL9, ITIN, I SKV ) 
IF(hC.eQ.l) 60 TO 10 
INO - 0 

DO 200 I ■ 2,NC 
«EAD(If4PUT,i)ICARD 
IF CEOFdNPUT)! 15,85 
6 5 NUH « MUn * 1 

IF C.NOT.tPRINT) 60 TO 100 
LINCNT « LINCNT ♦ 1 
IF CLINCNT.6T.55) CALL PA6ER(56) 
i<RITECI0UT,3)NUM,ICARD 
CALL NU«BER(IT0N,LISTR,2) 
IF CLISTRCl) .HE. I) CALL PER^OR(l) 
IF (LISTRC2).NE. ITIN) CALL PERRQR(2) 
IF ( LISTRd ) .NE.O) 60 TO 100 
LINCNT • LINCNT ♦ 1 
IF (LINCNT. GT.55) CALL PAGER(56) 
xRITEC IUUT,8) 
hUH • NUH - 1 
• 60 TO 30 
100 CALL NUfl85R(IT02,KCREA0,8) 
DO 110 J"l,7,2 

IF CKCREAD(J).£Q.O) 6C TO 110 

IF (f<CD(KCREAD(J + l),5).E0.0) 60 TQ 102 

LINCNT - LINCNT ♦ 1 

IF tLlNCNT.6T.55) CALL FAGER(56) 

K • (J+l)/2 

VRITE(I0UT,11)K 



RDTASK 

•iDTASK 

RDTASK 

R3TASK 

ROTASK 

RDTASK 

RDTASK 

RDTASK 

RDTASK 

ROTASK 

RDTASK 

RDTASK 

ROTASK 

RDTASK 

RDTASK 

RDTASK 

RDTASK 

ROTASK 

ROTASK 

ROTASK 

RDTASK 

ROTASK 

RDTASK 

RDTASK 

RDTASK 
RDTASK 
RDTASK 
RDTASK 
ROTASK 
ROTASK 
RDTASK 




S-^ILL vAUe Z^Rl - This jCC 



102 CG^TlMjc 

IF (KCi^EAuCJ+D.KE.O) GC T'.. lOt) 

IF < .N3r.LPRINT) GC TC liJ 

LivCM » LINC^T + 1 

IF ( LISCNT.GT.55 ) CALL PA3fc,^(56) 

WRIT£{ IUUT»5)KCi<bAD( J) 
5 F0R.1AT(* knOwLlDGE CATEGURY *,lb,* hA 
ILRENCL <,1LL 3E IGNORED*) 

GO TP no 
105 CJNTlNut 

LS « SEARCH(-<CRE AD( J ) » UO»KCd'JFF ) 

IF (LS) ISKVdND) « NCR6Aj(j + l ) 

IF (Li) Gl Tu 110 

IF ( .M T.LPRIM) oi. TL 110 

LINC-I « LI^CM ♦ 1 

IF aiNCM.GT.55) CALL FAGtR(5fc) 

WRITE(lubT»'f)'<CK£AD( J) 

FORMAK* <N3»,LEDGE CATEGJRY ♦^IS,* HA^ ALRtAJi AP■^tAPEr^ is [HP [A 
ISK - JrilS KCL^RE'.Cfc •ILL 3£ IG\.,Rf3*) 
110 CCNTlNut - — 

200 ClNi IK-Jl 

IF (^EwTS<) CALL ^^iM L I S 1 ( In J , -^C OF f- > I T I N , I S v * 

&(.. Tl 10 
END 



-^OTASk 
ROIASK 
KOr ASK 

ROT ASH 
ROT ASK 
RDTaSK 
ROT AS^ 
ROTASK 
RDTASK 
^0^ A.S-^ 
ROTA?K 
ROr A^K 
ROFASK 
RDTASk 
kDT AS< 

RDTA5'< 

kDF ASK 

K 3 r A i.. K 

K3TAS< 
RDTASK 
K 31 A NX 



h.j8R :^.TI^t M-»- =c M 10 jL, L IS T » L I W ) n.-l- 

CC'^vf /IJ/I^PL^, I-LT,ITA5E, ICAROCoC) ,LInCnT,HjT^K{ UOU) £-^Lko 

CI«'E\SK.% ICnL(r'Ll:f,2),LI. i (MI ST) .-'^^ 

LATA Ic3L/10H /» I /UHtTtttttr-rT/, :0/iCH / i -'P^H 

DO 100 I«i,NLnT r- 
J • ICjLd.U . 



LIST(U - 0 
IF (J.Eti.2) GO TO 5 
L « J - 1 

IF (ICAROCU .NE.IR ) GO T3 30 
5 L ■ K ♦ 1 

IF { ICARD(L) .NE.IR ) GO TO 40 

DO 10 L"J*K 

IN ■ ICARD(L)-1R0 

IF (ICARD(L).EQ.IR ) IN ■ 0 

IF((IN.LT.O) .CR.(IN.GT.9)) GO TO 20 
10 LIST(I) . LIST(I)*10 ♦ IN 

GO TO 100 
20 LIST<I» ■ 0 

L ■ L - 1 

WRITE<I0UT,1)(IBL,M"1,L),IAR,(ID,N"L,79) 

1 F0RriAT(14X,81Rl,* DIGIT ,iUST APPEAR HERE*) 
LINCNT ■ LIhCKT ♦ 1 

IF <LINCNT.GT.55) CALL PAGER(56) 
GO TO 100 
30 LIST(I) ■ 0 
L « L - 1 

WRITE (I0UT,2 )( IBL,M-1, L), lAR, (I0,N"L,79) 

2 FORiATi 14X,81R1,* BLANK MUST APPEAR HERE*) 
L INC NT ■ LINCM ♦ 1 

IF <LINCNT.GT.55) CALL PAGER(56) 
GO TO 5 
40 L ■ L - 1 

>«RITE(ICUT,2)(IBL,M"1,L), lAR, {I0,N"L,79) 
LINCNT - LINCNT ♦ 1 
IF (LINCNT. GT. 55) CALL PASER(56) 
100 CONTINUE 
RETURN 
END 



NUMBER 
NUMBER 
NUMBER 
NUMBER 
NUMBER 
NUMBER 
NUMBER 
NUMBER 
NUMBER 
NUMBER 
NUMBER 
NUMBER 
UMBER 
NUMBER 
NUMBER 
NUMBER 
NUMBER 
NUMBER 
NUMBER 
NUMBER 
►iUIBER 
MUHBER 
NUMBER 
NUMBER 
NUMBER 
NUMBER 
NUiBER 
NUMBER 
NUMBER 
NUMBER 
NUMBER 
NUMBER 
N JKBER 



SU3R'.uriNt ObRRCPd) 

1 N<T,^rAB(36000),.^TAS^,rAXrA8,f•A>^C ^ M . JU ) , I W( 700 » , 

CC»"<3N/CuLS/iTOO(3,2),no>.(2,2),I roi(16,2),iTi:(3,2) 

CCMhC■^/IC/INPL^,IL5UT,ITA^E,ICARD(dO),LI^CNI,LoT = R(1^00) 
CCM^«oN/iKlLLS/KEY».D( 16 ) , KE Y ALtt ( 16 ) 

Cn-MO.s/JELIS r /NOEL, LJtL( 700), KLT.Fh 
COMIG N / L ,1G / TT 1 , L N OR M , L Pft I •<T 

O.^'Cn/TCARC/ITOCN, ITL^,,s'C 



lcgical tti,lnli^.'',l;'ri.'><t 

LCGICAL S cARCH, L i^tNciP^T i*' 



CI'*t^:>I js <Ci<LA0(6) 
CnENbi IM ■(Ct3LhM70u),I->^v(7 00) 
OncNilu 4 LlST(3),LI5Tit(2) 
DI^fc^'SK^ f'<LIM(700) 
DI"<ESii:,N IF*AT(3) 

£ '. 1. 1 V A L t x C c ( L rw ( 1) , i T D C ^ ) , ( L I n -< { 1 ) , P ^ L i ^ [ i i ) ) 



PEKKfV 
^tRR?R 

EURO 
E iLRG 
c JC-r. 
E3CCM 
EDCr" 
E VLRG 
E jCC* 
EDCv.'* 
E JC^^" 
E-'iLRG 

EJCO^ 

c 0 c : M 

EDClM 
t3C " 

£OCi " 
E 3C. 
c \LPG 
cOCi *• 
E ..LRG 
c}C 



CAFA I.,L/lh I AK/lht/, Ij/i H-/, If. Ik r/o/ 

1 rjf) 



*• C K K '^ 



LINCNT . LINCNT * Z ^ 
IF <LINCNT.6T.55) CALL PA6ER(56) 
60 TO (10»30»20)I 
30 WRITE(IQUT#1) 
1 F0RHAT{18X*6Htttttt,72(lH-),* WARNING*/ 
^ETURN*^"^^ ^^^"^ DIFFERS FROM ENTRY JN SU»1MARY CARD*) 

10 hRITE( I0UT*2) 

^ J°J3JJ<^5X,2Htt,80(lH-),* APPARENT CARD SEQUENCE ERROR*} 
20 WRITECI0UT*4) 

20R THIS TASK UNTIL A (TOO) CARD IS FOUf^D*) 
R 1 1 UKri 
END 



PERROR 

PERRCR 

PERROR 

PERROR 

PERROR 

PERROR 

PEi^ROR 

PERROR 

PERROR 

PERROR 

PERROR 

PERROR 

PERROR 

PERROR 

PERROR 

PERROR 

PERROR 



10 

i:o 



SUBROUTINE PUTLI ST (N, L 1ST, ITIN, ISKV ) 

lNEXT,«TA8(36000),NTASK,MAXTAd,MAXKC "wyui, 
DIHENSION LIST(I),ISKV(1) 

DATA nk%\f,ZQizinm%i,n^%^'iQinmmmmQf 

IF (N.6T.MAXKC) CALL ABCRTC 20HISK V ARRAY OVERFLCW ) 
100 I«l,N 

(ISKV( I) ,£Q,0) GO TO 100 
(IBOT.EQ.O) GO TO 20 
10 J-1»IB0T 

(KC(J).EQ,LIST( I)) GO TO 5>0 
CONTINUE 
I80T ■ IBOT ♦ I 



DO 
IF 
IF 
DO 
IF 



PJTL 

E'lLRG 

ENLRG 

PUTL 

ENLRG 

ENLRG 

PUTL 

f'UTL 

PUTL 

PUTL 

PUTL 

PJfL 

PJTL 



ERIC 



170 



U ( ItJ.jI.GT.'^AyKC) CALL kUt^UZOHi^C A^RAI u^crFti. 
!<C(IiLT) • LIST(I) 
u • I8uT 
ITCI» • NEXT 
6J TO 60 
50 K • KfT(J) 

ITCP - AMu(hASK20,SriUT(fiTAd(O,20)) 

MA8(K) . JK(SHI>- r(NEX r><.D) , ANU(MTAh(K ),MAjn40) ) 
60 KFSEC(J) . KFREit.1) ♦ 1 

^TAe(^Exr) - r.^csHim n.j-»,^o),:iriiFT(iih,/(i),^o) ,itin) 

NEXT « NEXT ♦ 1 

,r,n jff .t^txT.&T.»«AXTAB) CALL Ad -R T( 2 OH T Ae Lt A^RAr . vt^FLC^ 
ICO Cu^TlNJt 

RETURN 

END 



SNLkG 
PjU 
.^JFL 
jfL 
^■JTL 
t'uTL 

E >«LK& 

r^UfL 

-'-•TL 
lnLkG 
PJl L 

PLIL 
^jTL 
''.ill 



LVtRLAr{ EOIT»3,0) 
FRuGRAI iCKT 

Cl,M^:N/TAt Lt^/IDl.■T»IrTAe{700 ),^C( 70C),hFREj{7CJ),KPT(70C), U(700 ), 
1 ^<T,^IAB(3600G),^TA5K,^A<TAa,hAX•<C 

CCfMf.f /CvJLWn0G{3,2),n0M(2,2), IIOl (lo,2),I rC^(3,2) 

Cut.*^uN/I J/INHT» ICl!r» ITAr'E, ICARDt60),LlNC,T, LliT-^^ (1^30) 

Cn.-^jN/iMLLWKtr*0(lo)MEyAL:i( 16) 

CLM^ >/jELl'>]/h DtL,LDtL( 700 ),-(i f.-.FF 
Cij^lJ,WLTG / ni>LN.;h!^,LKf<'I vl 
C■.^ '.N/ ICA^D/ITCL .,1 riN, ,J 
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t JC'J*'' 
E^LRG 
E XLKG 

enlkg 

EDCr " 
E JCs" 
c L 
c 'iL K G 

c JCr * 
tJC i'^ 



ERIC 



LOGICAL TTl#LNORH#LPRIN.i 
LOGICAL SEARCH, L$,NEi<T$K 



OIHENSION KCREA0(8) 
OINENSION KCBUFF(700), ISKV(700) 
DIMENSION LIST(3),LISTR(2» 
OIHEKSIOH PRLIST(700) 
OINENSIQN IFHATO) 

EQUIVALENCE ( L IST( 1 ) , I TOCN) , ( L ISTPR ( 1) , PR L ISK 1) ) 

DATA MASK20/3777777B/ 
CALL SHLSRT(KFREQ# I80T# IW) 
00 10 I«1«IB0T 

J « iw(n 

IF (KFREQ( Jl .GT.KUTOFF) GO TO 20 
10 LS • SEARCH(KC(J),NDEL#LDELJ 
20 IF (UDEL.GT.l) CALL SHLSRT ( L OE L#NDE L ) 

CALL INPAGE(IBOT) 

00 100 I-]i#IBOT 

CALL PKCd) 
100 CONTINUE 

CALL INPAGE(IBOT) 

DO 200 I«1#IB0T 

K - (IBOT+1) - I 

J « Ih(K) 

CALL PKC(J) 
200 CONTINUE 

CALL SHLSRT<KC* IBOT*Ik) 

CALL iNPAGE(lBCiT) 



ERIC 
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EDCOH 

EOCOM 

EOCOH 

EOCOM 

EDCOM 

EOCOM 

EOCOH 

ENLRG 

EDCOM 

ESLRG 

EDCOM 

EDCGM 

EDCOM 

EDCOM 

ESLRG 

SORT 

SORT 

S3RT 

SORT 

JiORT 

SjRT 

SORT 

SORT 

SORT 

SORT 

S3RT 

S3PT 

SORT 

SORT 

SORT 

S3RT 

SORT 

SORT 



300 



350 
<»00 



1 



<»10 

2 

<i20 

430 



TJ <i3C 



GC rn 3ijo 



Dt. 300 I»l,IaC'T 
J • U( I ) 
CALL PKC(J) 
CCwTIfUE 

CALL iNPAoEdbOT) 
ITCP « 1 

IF (NDtL.EQ.O) Gt 
DO <i00 I-l, IBCT 

0 > Im(X) 
ME-l? = <C(J) 
DC 350 S-ITjH^NDEL 
IF (KltrtP.NE.LOLL(K) ) 
CALL PKC(u) 
In(I) > 0 

LDEL(K) = 0 
HOP - < + 1 
GC rc <iOO 
CUNTUUt 
CONTINUE 

1 FLAG « 0 

0( <i20 I»1,N.DEL 
IF (LOEL(I) .6U.0) 1 3 420 
IF ( IFLAG..'Mfc .0) GU TJ 410 
k^RITEd'jJT*!) 
FJRMAKIHO** LR^i^ - 
lERE ,\LT F^U>0 In TUt 
IFLAG « 1 

wRITE(IJuT»2)LD£l( I) 
F:i'<»'A1( 20x» 18) 
C jmTINOE 

CALL IN^AG£ ( HOT ) 
Cj 50C I»l, IBLI 
IF(£iN(I).tC.U) Gi T'.i 5 0C 



- THE 
i iFur 



r J L L I *• I N 6 
DATA*) 



KCb „tKE rr ifc DELETED, BUT 



^ T 
.vIRT 
i JRT 
S l-^T 
S IRT 

> IkT 

> LkT 
<-j.<T 

rjKT 
i :r t 

i IRT 

s:'«T 
: :rt 

JRT 

> 3RT 
:kT 
J^T 

^R^ 

:rt 

•<T 

;:)rt 

.v.'iRT 

y.'RJ 
ijRJ 
. 'fif 
i JKT 

> JRT 

VT 



5 
J 
J 

'i 
:> 
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SORT 



INO . IND ♦ 1 

IW(INO) - Uil) I'yil 

IF (M00(IN0,56).EQ.0) CALL PA6ER(57) cop! 
IF<KC<J).LE.16) 60 TO 450 

WRITE(I0UT*3)IND,KC(J),KFRE0(J) cnei 

3 F0RHAT(8X>I4,3X, I8,2X,I<,) ^hbt 
GO TO 500 ^^J:l 

450 K - KC(J) l^:! 

WRITE(I0UT,4)IND,KErWD(K),KFRE0( J) ^hbt 

4 F0RMAK8X, I4,3X,R8,2X.I4) tno! 
500 CONTINUE t^^^] 

CALL INPAGECIBOT) tnor 

DO 600 I-1,NTASK J^eJ 

READ(ITAPE,5)ITTAB(I),(LISTPR(J), J-1,5) InVr 

5 F0R>'AT(I5*5A10) ^^^ll 

6 FORMATdOX, I5,6X,I5,5X,5A10) li.l 
IF <MOO(I,56).EQ.O) CALL PAGER(57) Z'iVj 

600 WRITE(I0UT*6)I^ITTAB( I),(LISTPR(J),J-1,5) cnpT 

l.RITE(I0UT,7)NTASK, IND Ji^l 

'i:SmEsiy)'55J°;:MD S^J^B^r ^^^^^^^^^^^^''^^'^O,* . 3F V SORT 

REWIND ITAPE l^ll 

IFMATCl) . 10H(10X,16F4. ,;^:J 

IFHAT(2) - 10H1/(16F4.1) c^nj 

IFNATO) - lOH) liy. 
WRITEi ITAPE, 8)IFhAT 
8 FORMATOAIO) 



XNORM ■ 0. 



SORT 



LAST - NXT-1 ^°JJ 

IF (LNORM) XNORM « S0RT(0.5) p^, pr 

650 00 700 I-1,NTASK Xlg^^ 

DO 680 J»1,IND .i^^j 
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SV ■ fHOli}' 

K » U(j) ' •'^T 

K • K P T ( K ) -' ]^^ 

N • iriIFT(MTAe(K ),-*,J) ^'""V 

660 K • SHlFr(MTAb(h ) ,-^0) c^-LRG 

« KaAT(AviD(MASK23,ShIfr{MrAB( v.),-20))) ♦ J.l '^'.'■-^ 
IF (LNuKD ;>K » S0RT(6< ♦ 0.i>) 

IlN » Ai>«0{/1f AB{N),MAS(<2C) tXL^G 

If ( IH.Ew.inAbd) ) GJ TJ 67J ^"-^^ 
IF (K.Lt.fO Gf Ii G6J 

f « K J 

GC lu 660 

670 Srf « SK ^ "^^ 

o80 PKL151 (j) » SV c^-LkG 

9 Fj«''AUiX,I7.16FH.l/{i5F^.l)) !:.'^J 

700 I.RnE(ITAPE,9)ITTA&{ I),(Puii r{o),j-l,IsD) ' 



.'JS-^r^LTI'viE PkC(I) 
CL»•^"•,^/:.<Io/LI.^iK^ 



i'^"I^v^^''?'iL'fi?''^:^^^''^'^°°*''^^ ^^^''•'^■'''•^'^^^>''^°'rt700),I.( 700), EmJkG 



J jKl 



Kb 

NL RG 



Cf* >/C'jLWI10G( J,2), ITuS(£,Z), I TOU 16,2), 1102(8,^) 
CG**^ JN/I l/I ^FUT, Ii'.Ui, ITAPc, I CAki)( 30) ,L UCM , LIi,T''^( l^JO) lll'-il 
CC1'<0N/:iKlLLS/KtrWD( l6),KtYU3i lo) ^^C 



17;") 



ERIC 



C0MH0h/DELIST/N'^FL,LDEL(7OO),KOTCFF 
C0H«0N/L0G/TT1#LN0RH,LPRINT 
C0MMQN/TCARD/ITDCN,ITIN# .4C 



LOGICAL TT1,LN0RM,LPRINT 
LOGICA. SEARCH#LS#NEWTSK 



OIHENSIQN KCREAD(8) 
DIMENSION KC8UFF(700),ISKV(700) 
OIHENSIQN IIST(3),LISTR(2) 
DIHENSION PRLIST(700) 
OINENSION IFMATO) 

EQUIVALENCE < L 1ST ( 1 I, ITDCN) , ( L 1ST PR( 1), PRL IST( 1 ) ) 



ILF - IH 

IF (hOD(LINSKP*20).EQ.O) RF - 1H + 
taRITE<I0UT*3)ILF 
3 F0R«AT(A1) 

J - LIMK(KPT(in 

K • C(J-l)/20)*2 

LINCNT - LINCNT ♦ K 

IF (LIHCMT.6T.55) CALL PAGER(^6) 

IF (LINCNT. EO. 2) LINCnT - K 

LINSKP - 4 

lASTER ■ IH 



ERIC 
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EOCOH 
ENLR6 
EOCOH 
EOCOH 
EOCOH 
EOCCH 

EOCCH 
EOCOH 
EOCOH 
EOCOH 
EOCOH 
EOCOH 
EOCOH 
ENLRG 
EOCCH 
ENLR6 
EOCOH 
EOCOM 
EOCOH 

EDCQH 

PKC 

PKC 

PKC 

PKC 

PKC 

PKC 

PKC 

PKC 

PKC 

PKC 

PKC 

PKC 

PKC 



IF (\OEL.tQ.O) G'J T.J 30 
DC 10 L « 1#N0£L 

IF (LCEL(L) .bW.KCd)) lASFdK » IH ♦ 
10 CCNTINub 

30 IF (KC( I) .LE.16) &G ru tfO 

VRUEHJUUDI, IAifER,KC(I),K-FREO( I),(LISrPf<(L ),L = 1, J) 

1 F0RMAT(1X,I4,2X,A1, 18, 2X, 1 0 ( 2X , 1 4 , 1 H/ , F3 . 1 ) 

1 60(/Z2x,10{2A,H,iH/,F3.1))) 
RETURN 

<.0 K • i^Cd) 

WRITE(lr,Jl,2)I, lA::TE-<, KE r^D ( K ) , KF RE3 ( I ) , ( L I PR ( L ) ,L » 1 , J ) 

2 FORHAK ix, I^f 2X,Al,K8,2Xi.H, 10(2X, I4,iH/,F3.1) 

1 60(/22X,10(2X,M,lH/,F3.1))) 
REloRN 

END 



?kQ 
f*<Z 
OKC 
•><C 

PKC 

MC 
<>KC 
PKC 



FUNCHQN LlNK(ItNO) 

C0»<M:.N/rA3LFb/IB.'.l,inAB( 700),KC( 70O),KFRtU7O0},K,>T(70C),U( 700), 
1 N<T,^rAB(3b000),^TAS^,•'A<rAB,rA)<KC 

C J-'--JN/CjL b/IT09(3,2), irON( 2,2) ,1101(16,2 ), ir02(8, 2) 

cj'^^j^/nnH?vi,iii,j,ijk^t,iCi,RO{ so) ,linc,nt, lister i i^oo 

CJ«'.''>/>,<lLL.s/KtV,D(lfc >,K5YALtj( 16) 
C01'^CN/ijFLljr/NJFL,L JfcL( 700) fMT-FF 
COMMON/ L jG/Tri,L NCR*',! I Mf 
Cu.'^f^'JS/TCARJ/ ITL^C J . 1 f I m,nC 



L INK 
EDCr»' 
E JLRG 
t>tLRG 
cDCr.'* 
E jCO** 
fcDCL'- 
t v-LRG 
t)Cl ^• 

cDc:- 

E ^LPG 

i oc'y 

tDCi - 
tJCi.- 

i. J C ^ 



ERIC 



LOGICAL TT1,LNDRM,lPRINT 
LOGICAL S ARCH,LS,NE*<TSK 



OIHENSIUN KCREAO(d) 
DIMENSION KC8UFF(700), ISKVC700) 
DIMENSION LIsm),LISTR(2) 
DIMENSION PRLIST(700) 
DIMENSION IFMATO) 

EQUIVALENCE tL 1ST ( 1 ) , I TOCN) , ( L ISTPR ( 1) , PRl. i^T { i ) ) 
DATA nASK20/37777778/ 
I ■ 0 

N • SHIFT(MTAB(IEN0),-40) 
NEXT ■ 3HIFT(hTAB(N),-^0) 
I ■ I ♦ 1 

LISTPRdI ■ AK0{«ASK20*fTAB(N) ) 
I ■ I ♦ 1 

PRLISTCn - FL0AT(AN0(SHIFT(f1TA6(N>,-20),f1ASK20)) / 10.0 
1. 1 NK • I 

IF (NEXT.LE.N) RtTUR'4 
H ■ NEXT 
GO TO 10 
cNO 



EDCOM 

tocor 

EDCDM 

EOCGM 

EOCCH 

EOCOM 

ENLRG 

EOCC»« 

ESLRG 

EOCCM 

EOCOM 

EOCOH 

EOCOM 

EI4LRG 

LINK 

LInK 

ENLRG 

ENLRG 

LINK 

ENLRG 

LINK 

ENLRG 

LINK 

LINK 

LINK 

LINK 

LINK 
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SHLi.<T - 1 mr.RO/tSn\ lAdLt SjkT 



CALL 
CALL 



6HLiRT{ 
iHLSf<T( 



TABLE 
TABLt 



PTr( ) 



- CUtCr ( 

- iNDIRFCr 



In pLA; 
S:iRT 



) T-: 6E 



TAdLE - A.-^^Kdii LF TABLE ( ARRAr 
N . NUMBER uF •CftOS IN TABLE 
tI\1 CF A.N AR^Af Te BE UiFO AS A 

TAjLE ( DI-TcMSIoNED PTR(N) ) 



SORTED 
t>JlNTER 



IF THE THIRD 
SCRT WILL 3E 
3t U:>LO AS A 



?ERFrR?Jr/\'"p^y-^"'' ' ''''^'^ < ''LACE 
PERFvKffcO, Else THc ThIRO ARGJ^E»CT WRL 

iLRT WTi. «^ o^if ^^^"-^ INDIRECT (PCHr^R ) 



SHLS< T 

SHL'fOT 

SHL^^T 

SHL'JRT 

SHL^RT 

SHLSR T 

'•HL^RT 

SHL'fRT 

?HLSR r 

5HLSRT 

SHLSRT 

SHLSi?T 

iHL'JRT 

SHLSRT 

SHL<^RT 

SHL SRT 





t'JTRY 


SHLSRl 


6iS 


2 


SToO 


iA5 


SldA 




SAO 


X5 




PS 






S^l 


1 




SA2 


Al+31 






AO 




;a3 


A2 + 31 



KEiKiR^ AO 



5hl';rt 

^HL'^Rf 
5HL >Rr 
,HaSRT 

r HL T 

5.HL^iJT 
'^HLSoi 

^HL'-?r 
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ST31 



STB2 



STB3 



STBI 



SA2 


X2 


SAO 


Xl-1 


SA6 


STBA 


SB7 


X2 


S36 


X2 


NZ 


X3*STBi 


3X6 






1 


SB6 


X6 


S83 


31 


ZR 


B6,STB0 


S34 


37-36 


S32 


33 


SB5 


32*36 


SAl 


AO+32 


SA2 


A0+B5 


1X4 


X2-X1 


PL 


X<.,5TB3 


BX6 


XI 


LX7 


X2 


SA6 


A2 


SA7 


Al 


^ a *i 


32-b6 


GT 


B2,STB2 


SB3 


83 + Bl 


5,32 


33 


IE 


33*B<.,ST82 




STBI 


3X6 


XI 


S62 


36-Bl 


SAO 


X3-1 



AO ■ Fti<A'-l 
5 AVE AO 

N • N (LE^GTH) 

IF AN INDIRECT SORT REQUESTED 

1 = n/z 



J • 1 

RETURN IF ^1 « 0 
K • N-H 
I • J* 

L • i+i 

Ad) 
A(L) 

IF A(L) .0 . A( I) 
INTERCHANGE A(L) AND A(I) 

I a I-M 

IF I .GT. 0 

J « jfl 

I • J 

IF J .LE. K 

SET UP PCIwTE-^ TABLE 

AO * FwA-1 ^UIMTER TA3Lc 



5HI.5RT 

SHL^RT 

SHLSRT 

SHLSRT 

SHL^^RT 

SHLTRT 

SHL^RT 

SHLTRT 

SHLSRf 

SHL^RT 

SHL^nRT 

SHLSRT 

SHLSRT 

SHLSRT 

5HLSRT 

SHLSRT 

shl<;rt 

SHLSRT 

SHLSRT 

SHLSRT 

SHLSRT 

SHLSPT 

SHL<?RT 

SHLSRT 

SHLSRT 

SHLTRT 

SHI SRT 

SHLSRT 

SHLSRT 

SHLSRT 

SHLSRT 

SHLSRT 
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180 



ST3I1 



ilBI2 



5.T913 



5Ao 
ix5 
5o2 
SA6 
S2 

SX6 
AX6 
^b6 
5B3 

S82 
SB5 
iAl 
>A£ 
iA3 

U5 
PL 

3Xo 

L«7 

SA6 

iA? 

SB2 

31 

SB3 

se2 
tfc 



A6+51 
X3 

X6 + 61 
32-81 
A6*31 
32,»-l 

36 
1 

X6 

Bl 

B6,STBU 
37-36 

tJ3 

32+JJ6 

A0+&2 

AO+85 

XI 

X2 

X5,STdl 3 

XI 

X2 

A2 

Al 

62-b6 
b2,$TBI2 
33 + Bl 
d3 



;1 « ^/2 



J « 1 

IF C » 0 
K « 

I « J 

L » 1+1 
l>{ I) 
P(L) 



63,£,<»,br6I2 If J .L£. < 
3TBI1 



A(P«L)) .GT. A(P(n) 

I^fcRC''^A^G£ a <o (u 



I « i-r 

IF I .Lf. 0 

d = w + l 

I - J 



SHLS^T 
SHL«?.RT 

5HL$RT 

.SHLSRT 

?HL«i?T 

SHLSRT 

SHLSRT 

SHLSR T 

SHL5JRT 

SHLSRT 

5HLSRT 

SHLSRT 

SHL«JRT 

SHLSRT 

' '-^RT 

SriL«5RT 

SHLSRT 

SHLfRT 

SHLSRT 

5HL ?RT 

SHLSRT 

'.HL^RT 

SHL^..'T 

<^ <1<"RT 

5HL"^Rr 

<;HLrRT 

SHL SOT 

SHL"^-? T 

SHL'^PT 
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ST8I4 



STBI5 



3 A 7 


C T Q A 




CAT 


A ^ X Q 1 
A P TO X 


ADUKtSS Or A { 


S62 


AO^Bl 


P(l) 


SB3 


B2+B7 


LWA+1 OF P 


b AO 




RESTORE AO 


SXl 






SA2 


82 


CHANGE POINTER 


SB2 


62 + Bl 




I}«6 


X2-Xi 




SA6 


A2 




LT 


82#B3#STBI5 




EQ 


SHLSRT 




END 







TABLE ) 



SHL'^ST 
SHLSRT 
SHLSRT 
SHLSRT 
SHLSRT 
SHLSRT 
SHLSRT 
SHLSRT 
SHLSRT 
SHLSRT 
SHLSRT 
SHLSRT 
SHLSRT 
SHLSRT 



SUBROUTINE ABORT(SSfi) 
INTEGER MSQ(l) 

CALL SVSTPM(52,12H USER ABORT.) 
STOP 

ENTRf ABORTM 

CALL SySTEM(52,MSG) 
END 



AtOf^T 
APQPT 
ABORT 
ABORT 
AP^OPT 
AnoPT 
AR'JPT 
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APPENDIX C 



EDIT WITH THE PCVARIM PROGRAM 



HSMS uses the PCVARIM program to select the "solution* 



(number of factors) which best groups the skill and knowledge variables. 
PCVARIM is an abbreviation for Principal Components Fact9r Analysis 
with Varimax Rotation. 

Because the number, identity, and frequency of the skill 
and knowledge variables cannot be ascertained beforehand, because of 
the size limitations of the PCVARIM program, and because EDIT provides 
a logarithmic transf 07:matlon of the data, PCVARIM is always used by 
HSMS interfaced with EDIT.-*" 

DESCRIPTION 

PCVARIM is written in FORTRAN IV and was used in th^ Control 
Data Corporation's (CDC) 6600 computer at the Courant Institute of 
Mathematical Sciences of New York University. The operating systems in 
use during HSMS analyses were KRONOS and NOS (Network Operating System). 
PCVARIM was stored on magnetic tape in compiled and loaded form (i.e, 
in binary object code in non-relocatable form) and in OLDPL form. 



form of an OLDPL. A utility program, UPDATE, is used to transfer data 
to tape, correct data, or to generate temporary local files in the 
form of a compile file, x^hich is part of the input file for EDIT. The 

The EIDIT program appears in Appendix 3. 



The HSMS task data are transferred to magnetic tape in the 



C-l 



ERIC 




Inpc t file for UPDATE is magnetic tape in OLDPL form, and the output 
file is C=DATA- The input file for EDIT is DATA, and the output file 
is TAPE9 (with optional printed output). The input file for PCVARIM 
is TAPE9, and the output file can be printed or punched. 

The most attractive feature of PCVARIM is that all the fac- 
tor solut ions (i.e., the number of factors to be rotated) one wishes to 
examine can be requested with a single submission. As long as the num- 
ber of principal axis factors to be extracted is as large as the larg- 
est number to be rotated, any and all smaller number solutions can be 
called with little additional cost. Therefore, the user has all the 
solutions to be examined available at one tipe 

The tasks and the skill and knowledge variables enter 
PCVARIM in the order of the final internal numbering assigned b> the 
EDIT program. Therefore, the EDIT "dictionaries" are used to inter- 
pret the rCVARIM output. 



HSMS makes an inspection of the output as an error check 



Check that means and standard deviations seem ap- 
propri^^^e for the data. 

Check that all rotations called for are present. 

Check that no nutr/ocir in the correlation matrix ex- 
ceeds 1.0, that no factor loading exceeds .999, 
and that matrix, correlation and factor loadings 
are not all zeroes. 

As of September, 1977, the comment cards included in 
the program had not been updated to includa modifica- 
tions of tlie PCVAPIH Rotation Option Card. The user 
should follow the Instructions for structuring the 
input file which follows, rather than the comment cards 
in the program file (which are net reproduced here). 

C-2 
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as follows: 

I. 

2. 
3. 

Note: 




} 



STRUCTURING THE INPUT FILE: EDIT WITH PCVARIM 

In the set-up presented below, the NOS system in use in 
1977 at the Courant Institute's CDC 6600 is assumed. It is also as- 
sumed that the reader can refer to the UPDATE Reference Manual or a 
counterpart program. In the fo"* lowing instructions b denotes a blank. 





INPUT FILE FOR EDIT WITH PCVARIM 


Name 


Keypunch Cards (on 2 per box) 


Instructions 


Sys- 
tems 
Cards 


.EDIT With pcvarim 


Identification number and 
other infcmnation depending 
on system. 




USER( ) 


USER and user code. 




charge ( ) 


Charge card. 




HF\DER,EDIT with PCVARIM 


Optional to print out head- 
ing across a whole page. 




LABEL (OLDPL,VSN=T ) 


Identifies OLDPL tape. 




UPDATE(Q,D,8,C=DATA) 
or 


When selecting tasks using 
*COMPILE form of UPDATE. 




•JPDATE(F,D,8.C=DATA) 


Whf.n using all data in OLDPL. 




UT.TOAD(OLDPL) 






LABEL ( ^ROCRMS , VSN=T ) 


Identifies program tape. 




COPYBF ('VROG^^KS .EDIT) 


Copies program to local file. 




j:OPYHF(PROr,RMS , PCVARIM) 


Copies program to local file. 




UNLOAD V ^ROGRMS) 






SETTL(IOOO) 


Time limit. 




RFL( 200000^ 


Defines field length. 




EDIT. 






REtJIND (OUTPUT) 


Use only if no printed EDIT 
output is desired. Usually 
it is desirable tc have 
EDIT output. 




RFL(105000) 






PCVARIM. 






End of Record Card 






c-3 



in- 
put 
Cards 
for 
UP 

DATE 



INPUT FILE FOR EDIT WITH PCVARIM (continued) 



Keypunch Cards (one per box) 



*COMPILEb 

etc . , or 

*COMPILEb ,_ 

(option) 



In^ 

put 

Cards 

for 

EDIT 



End of Record Card 



TITLEb b b b b 



etc. 



NOLI ST 

(option) 



DELETEb b b b 
(option) 



CUTOFl-'b b b b 
(option) 



NORMALIZE 
(option ) 



Ins truct ions 



Calls tasks from OLDrL. Each 
card starts: *COMPILZ in Cols. 
1-8; Column 9 blank, followed 
by Task Code Numbers separated 
by commas; last entry has no 
comma. Tasks can be in any or- 
der. Consecutive Code Numbers 
can be called as last entry on 
card after comma or as only 
entry or card by writing first 
Code Number, then a period, 
then last Code Number of the 
series. A new card must fol- 
low. 



Needed even if no UPDATE car ds. 
Columns 1-5: punch TITLE; 
Columns 11-60: punch title of 
program, date, and any other 
special identification of 
this run. 



If listing of data and some 
error checks are to be sup- 
pressed. Usually selected 
with PCVAIvIM. 



One card for each individual 
skill or knowledge category to 
be deleted from input data (if 
not to be deleted by CUTOFF). 
Columns 1-6: punch DELETE; 
Columns 11-18: punch 8-char- 
acter skill code or 8-digit 
knowledge category code num- 
ber . 



End of Record Card 



To eliminate skill or knowl- 
edge categories at or below a 
selected frequency fro^ inpUL 
data. 

Columns 1-6: punch CUTOFF; 
Columns 11-12: punch selec- 
ted frequency ( isually 4 or 
more) left justified i\\ 
field. 



Usually used when EDIT is 
used with PCVARIM. 
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INPUT FILE FOR EDIT WITH PCVARIM (continued) 



Name 


Keypunch Cards 


(one per box) 


Ins true tions 


In- 








Title Card. Cols. 1-40 read 


put 








as one line ; Cols 41-80 read 


Cards 








as second line. Punch title 


For 


etc . 






of program, date, and any 


PCVA 








other special identification 


RIM 








of this run. 




Cols. 4-6 




— — 


Right justify in all fields 


Para- 








unless otherwise indicated. 


meter 








Number of ski 11 and knowledge 


Card 








categories to be treated as 










variables; up to 145. (HSMS 










uses up to 144.) 




Cols. 8-12 






Number of tasks to be treated 










as observations! un ^o QQ QQQ 




Cols. 17-18 


— 


— 


umber of principal axis fac- 










tors to be extracted; up to 










12. (Select the larpe^jt num— 










ber to be rotated.) 




Cols. 23-24 


— 


— 


Number of varimax factors to 










be rotated. (Select largest 










llUIIlL^wL VJCOXLvVJ, UUL \.\\J /^i. JclLcL 










than Cols 1 7-18 




Col. 36 


— 




Punch W If no pHH^^^nn^l rn— 










t a t ions . 










Punch 1 if additional rota- 










tions are wanted. 




Cols. 61-67 


T 


A P E 9 


Name of alternate input file 










when EDIT is used (left justi- 










fied) . 


Rota- 


Cols. 2-3 






Use if parameter card selected 


tion 


5-6 






additional factor rotations 


Op- 


8-9 






(1 in Col. 36). 


tion 


11-12 






Enter number of varimax fac- 


Card 


14-15 






tors to be rotated and list- 




1 -18 






ed; as many as are of inter- 




20-21 






est, in descending order in 




23-24 






the fields Indicated (from 11 




26-27 






down to 2 is possible), so 




29-30 






that each is less than the one 










before. Right justify. 




End of File Card 
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4 



16 



tVERLA Y(HC VARIf*,0,0) 

PROGRAM ^CVARlM{TAPE9,lNPJT,JJTPur,P0NCH,TAPEt?»lNPUT,TAr^66 

A - CjRRELATICN MATRK 

S - VARIMAX ROTATED FACTQRS 

CDMhQN // b2{145),Q3(14i),Q4(14i)) 
COHMuN // TIT(1^),F^T(14) 
INTEGER ^NR(ll) 

INTEGER lN#ObT 
DATA ^nR/11*0/ 

In « & 
JOT « 6 
10 RE^D(lN,500) TIT 

IF (EtjF(lN),^t.O) GC Tu 350 

RE A0( IN# 3 70)NVAk ^.Mf'E :p*N^AXF A, NRJUA,nvPCH, L>i Jk, C IT, I LF K 
IF iiriR .t'^.O) Go JZ 15 
READ{lN>16)(MNk(i),I»l,10) 
FJft^ AT( 1013) 
IrujR a 1 



77/09/15 



15 



^OUTPUT) 



IF (ILFN..4E.7L ) IN « 9 

kdAj(INf500) FM 
^RITE ( JUT, 39C) TIT 
WRITE( JuT, 370) 

1 ^VAK,NPb:p,\PA)^FA,NRjrfA,.v*vPCH,lf1LRfCftiT,lLFN 
W(<I1E( OUT, ^00) FMJ 

IF (Nrt ^P.Lb.O) Gl TJ 8C 



KR.jNQS 

PC VARI** 
PCV-^COH 

Pcv-cnM 

PCVARIM 
rCV ARI^ 
PCVARIM 

kronhs 

^R jNQS 
^RuNOS 

^CVARI 
PC VARI^ 
r^CVARIH 

^R.J^^? 
KRC^OS 

< R J^'OS 
^R..•^?3 

PC VAR IM 
^C V AP I M 



CORRELATIONS 
RAW DATA INPUT 



20 



30 
'^O 



50 



60 



70 



80 



85 




DO 20 I 
S(U1) < 
S(I»2) ' 
00 20 J 
A(bJ) • 
00 40 K 



• IfNVAR 
0.0 
0.0 

' IfNVAR 
0.0 

1»NPEQP 



REAO(IN#FMT)( >( I»3),i»l#NVAR) 
t^O 30 I • If^VAR 
SfltlJ - S(I,1)+S(I#3) 
SllrZ) ■ S(I»2)+S(I»3)**2 
00 30 J - 1« I 

A<I*JI • A{i,J)*S(I#3)*S(Jr3) 

CONTINUE 

PEOPL ■ NPEOP 

00 50 I « 1«NV/R 

Sn#4» • Sdiii/PEOPl 

Sil»5) » $QRT( (S(I,2)-PEaPL*S( I#4)**2)/PE0PL) 
00 60 I • IfNVAR 
00 60 J « 1>I 

A(I#J) » (A(I,J)-PEaPL*S(I»4)*S(J,<»))/{PE0?L*S(I,5)*S{ J»5)) 
00 70 I • lyNVAR 

00 70 J • 1,1 

A(J,n ■ &'I#J) 

WRITf (0UT,360} TIT 

WRITE(0UT,410)( I,S( I,4),S(I,5),I-1,MVAR) 

GO TO 100 

00 8 5 I » 1#NVAR 
REAOdN/FMTJ (A( I,J), J»l,l) 
00 90 I • l^SVAR 
00 90 J ■ 1,1 
A(J,n - A(I,J) 
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t>CVARIM 

PC VARI1 

PCVARIH 

PCy/ARI»i 

PCVARIM 

PCy/ARIM 

PCVARn 

PCVARIM 

PC VARn 

KPONOS 

PCVARIM 

PCVARIM 

PCVARIM 

PCVARIM 

PCVARIM 

PCVARIM 

PCVARIM 

PCVARIM 

PCVARIM 

PCVARIM 

rCVARIM 

PCVARIM 

PCVaRIM 

PCVARI M 

PCVARIM 

PCVARIM 

K«ONOS 

KRONQS 

PCVARI1 

PCVARIM 

PCVARIM 

KRONOS 

PC^^ARIH 

fCVARIK 

PCVARIM 

PCVARIM 



C CORRELAriQN uCTPUT 

C 

100 ASSIGN 110 TJ KKTW 
LL » 1 

110 IF UNWAR-LL)-9) 120,120»130 
120 ASSIGN 160 TC KKlv 

LS » hVAR 

60 TO 140 
130 LS « LL + 9 

140 taRITE(JUT»430) TIT» (J, J»LL,LS) 
OG 130 I > IfNVAR 

150 ViRITE(UUT»420) { I,{A( I, J) , J-LL#LS) ) 
LL » LL+10 

GO TO KKTV» (llClbOj 

160 St'H » 0,0 

00 170 I • 1,NVAR 
170 SUM m SUM*A( 1,1) 

uRITE (:uT»4yO) SL" 

c 

C PRINCI^'AL AXIS FAClQRi 

C 

IF (M-AXFA.EQ.C) GO TC 10 

CALL Hj* (NVAft,14<.,SPAXFA,A#\/,S»ri,w2»w3»-4) 
►P » NPAXFA 
CO 130 J » 1,KP 
R • S0i<T(A5S(V( J))) 
0^j IbO I " l#NVAft 
130 Ad, J) . R»S(I»J ) 
fRITE{0uT#440) Til 



ERIC 



190 



PCVARIM 
PCWARIM 
PCVARIM 
PC VARII 
PCVARI'^ 
PCVARIM 
PC VARIM 
PC VARI»» 
.JCVARI- 

"CVARIM 
KRJiiOS 
PCVARM 
PCVARIM 

PC WARI1 
PC VARI". 
PCwAR !M 

fCVARIM 

PCV AR If 
PCVAkI 1 
PCVAR IM 

PCV ARIM 
PCWAR 1*1 

-Ovarii 

('C VAR I M 
PC VARIM 
PCvARIM 
PCVARIM 



iHRIT£|.JOT»<i50) (I,v(I), I-i,NvAR) 
SU»' • 0.0 
OQ 190 I « 1,KP 
190 S'JH • SuM*V{I) 

WKlTE{auT»460) KPfS'oM 
ASSIGN 200 Tu KK 
LL - 1 

200 IF {KP-LL-9) 210»210#220 
210 AiSIGN 250 TO kk 

L3 » KP 

Go T3 230 
220 LS • LL+9 

230 fcKITEi ]UT,470)TIT, { J, o.LL,Lo) 
OC 240 I»1,K<VAR 

240 WRITE (uJT»420)( I,(Af I, J), j.LL,L 
LL • LL*10 

GO TQ KK» (200,250) 

C 

C VARI»1AX FACTORS 

C 

250 IF (NRofFA) 260»10,290 

260 IF (NPAXFA.LE .0) 60 FU 10 
00 270 I » i,NPAXFA 
IF (V(I) .LE .CRIT) Gli To 260 

270 CONTINUE 

280 NR • I-l 

IF (Nk.lE.O) GC' to 10 

290 IF (nROTFA.GT.NFAXFA) GC TC 10 

NR » NkOTFA 

If CNR-1) 10,300,310 
300 W8ITE(ObT,380) 

GC TO 10 
310 OQ 320 I » 1,^4VA^< 

DO 320 g « l,Nk 
320 S(I#J) > A(I,j) 



i^C VA^ I M 
o:\/ARI«< 

?: vARi « 

< Run Pi- 
pe vAR 1 
PCVARl ^ 
•»CvARI'« 
PC VAKI 
PCVARIM 
PC VA« H 
PCVAKI *1 

kR'jn':'-' 

P: VARI"* 
PC^ARI ^ 
PCV ARIi 
PCVARIM 
PC VARI1 
PCrfARI^ 
PCvARIM 
5CVARM 
PCVARIK 
PCVARIM 
PCVARI^I 
PCtfARiM 
PCVARIM 
=>C VARIM 
PCVARIM 

\RjN!''$ 

PC V AR I 
^Ci/ARI^ 
PCVARI-1 
PC VARI '* 



325 OQ 330 I « 1,NVAR 

00 330 J » 1#NR 
330 A(I*J) « S( I» J) 

00 3<»0 I » l,NVAk 

H{I) • 0.0 

00 3<»0 J » 1,NR 
3<»0 H(I) » ri(I) + A( I, J)*»2 

Call varim ( nva.<,nr,.^voch) 

If( f1NR(If<ljR) .tQ. 0 ) GO T3 IC 
NK « MNR(IhDR) 
IfJR » IMOR-H 
GO TO 325 



360 FOR>«AT ( IHl, 20X# 13A6, A2 ) 
370 f0RMAl(6I6,F12.6,*T61,A7) 
380 FORMAT ( IH 1, 55HR3TAT ION nJT 

lED } 
390 FORMAT 
<»00 FORMAT 
<»10 FORMAT 
<.20 FORMAT 
^30 FORMAT 
^'fO FORMAT 
450 FORMAT 
460 FORMAT 
470 FORMAT 

1/) 

480 FORMAT 
490 FORMAT 
500 FORMAT 



DONE SiNCt CnLY ONt FACTTR TU 8E RvTAT 

( IHl, IIHINPUT CAROJ//, 13A6#A2) 
(IH ,13A6,A2) 

( lH0,lfcX*4HMEAN*16X,4HS.J.//(lH ,Io,2F18.6) ) 
(IH ,13,1 OF 11. 4) 

( IH1,2X, 12HCURRELATIuNS,10X, 13A6, A2//lh ,2X,1CI11//) 
( 1H1,20X*13A6,A2//1H , 1 IHE I GEN V AL UE S/ / J 
(IH ,5X,I3,5X,F12.6) 

(1H0,12HSUM QF FIRSn3,15H EIGE'^VALUtb « F12.6) 
(1H1,2X,22HPRINCIPAL AXIS F AC 1 OR S * 2X * 13 A6, A 2 / /I H ,2X, 10111/ 



( lH0,12HThE 
( 13, ID 
( 13A6, A2) 



TRACE IS,F18.6) 



350 CONTINUE 
END 



>»C>/A«IM 
PCrfARI" 
•'CV* ARI^ 
PCVARH 
PCVARIM 
•>C VARIM 
i>CVARI^ 
PCVARIM 
OCV ARIM 
PCVARIM 
PCVARI 1 
PCVARI1 
: v-VARI" 
PCVARI'4 
OCVARM 
PCV ARIM 

PCVARI*1 
PCVARIM 

PCVARIM 
PCVARI** 
PCV ARH 
PCVARI-i 
PCVARIM 
PC VARI« 
PC*/ ARI»« 
PCVARIM 
PCVARIM 

PCVARIM 
PCVARIM 

PCVARIM 
PCVARIM 
PCVARIM 
PCVARIM 
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SUBROUTINE VARIM(NRCi<S#NCaLS,NVPCH) 

A - CORRELATION MATRIX 

S - VARIHAX ROTATED FACTORS 

COHHOM// A(:44,144),S(144,12),H(145), V(145) 
REAL R00T(10)#VAR(10)#CUNV(10) 

VARIHAX FACTOR ROTATION 

OUTPUT COMHUNALITIES AND NORMALIZE FACTORS 

SUM "0.0 

00 10 I • l^NCOLS 

10 suH • sun^v(i) 

WRITE (6#260) NCOLS#SUM 

URITE (6#270i 

SUH - 0.0 

DO 20 I • 1#NR0WS 

WRITE (6»280) I#H(I) 

SUN • SUM>H(I) 
20 Hd) • SQRT(H(I)) 

WRITE (6>290) SUM 

DO 30 J • l^NCOLS 

DO 30 I • 1#NR0WS 
30 A( 1,4) • A(I# J)/H(I) 

KROT - «NC0LS*(NC0LS-1) )/2 
40 KX • 1 

KZCT • 0 

COMPUTE SUfIS A,B,C AND 0 



VARIMP 

PCV-CCM 

PCV-CQi 

PCV-COi 

PCV-COM 

VARIM2 

VARIM2 

VARIM2 

VARIM2 

VARIM2 

VARIM2 

VARIM2 

VARIM2 

VARIM2 

VARIM2 

VARIM2 

VARIM2 

VARIrt? 

VARIM2 

VARIM2 

VARIM2 

VARIM2 

VARIM2 

VARI»<2 

VARIM2 

VARIM2 

I/ARIM2 

VARIM2 

\/ARIM2 

VARIM2 

VARIM2 



ERIC 
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50 KY « KK+l r/ARI-2 

60 CONA • 0. VARIM2 

CONB • 0. VARIfl2 

CQ.'^C • 0. 1/ARIM2 

CUNO « 0. VARIf*2 

DO 70 I • l#NROWi VARI««2 

U- (A{I#KX)*A{I#KY))*(A(I,<X)-A{I»KY)) VARIf2 

V« 2.0*A{I#KX)*A{ I,KY) >/ARIM2 

CONA - COnJA+U VARIM2 

CONB - CONB + V VARI>^2 

CONC • CuNC*{L*V)*{ U-V) VARIM2 

70 CONO • C0ND + 2.0*L*V \/ARI»»2 

FLN « FLCAKNRCWS) VARI««2 

TNU** • CQN0-(2.0*CCiNA*CCN'li) /FLh \^ARIH2 

TOEN • CJNC-{ (CLNA+CaNa)»(CCNA-CQ^3 ) )/FLN VARIM2 

ANUM - ABS(T^U^•) VARIM2 

ADEN « ABSCTDEN) \/ARIM2 

IF (AMJM-ADEN) 90,30#100 VARIM2 

80 IF (ANUM.EU.O) Gij TO 180 \/ARIM2 

GO TO 105 VARIM2 

C UARIM2 

C ^UM LESi> THAN DEN VA.RIri2 

C VARIM2 

90 TAN^T ■ AnJM/ADEN VAKI12 

IF (TAN4T.LT. 0.06993) GO TC 190 i/ARlM2 

COSM - 1.0/$0KT(1.0*TAN4T**2) VARIM2 

SIN^T - TAi^t'fT^CCS^T V4RIM2 

GU TO 110 \/AkIH2 

C y/ARIM2 

C MJM GREATER THAf DEN (/ARIM2 

C VARI*12 
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100 CTN4T ■ ADEN/ANUrt 

IF <CTN4T.LT. 0.06993) GC TG 200 
SINU • 1.0/S0RT(1.0*CTN4T**2) 
COS^T • CTN'fT+SlN'H 
GO TO 110 

NUM EQUAL 06N# BOTH NON-ZERO 

105 SIN4T • 0.707107 
CQS4T - SIN'tT 

COMPUTE ANGLE OF ROTATION 

110 C0S2T - SCRT( 0.5*0. 5*C0S<»T) 

SIN2T - SIN'fT/CZ.O^COSZT) 

COST ■ SQRT{0.5*0.5*C0S2T) 

SINT « SIN2T/(2. ONCOST) 

IF CTOEN.LT.O) GO TC 120 

COSPHI ■ COST 

SINPHI - SINT 

GO TO 130 
120 COSPHI - 0.707107*(COST*SINT) 

SINPHI - ABS( 0.707107*(C0ST-SINY) ) 
130 IF (TNUM.LT.O) SINPHI - -SINPHI 

ROTATE FACTORS 

1<»0 00 150 I ■ :#NROWS 

U » A( I,KX)*CD5PHI*A( I,Kr)*SINPHI 
A(I,KY) ■ -SINPHI*A( I,KX)*A(I,Kr)*CaSPHI 

150 A(I,KX) » U 

160 IF (KY.EQ.NCQLS) GO TO 170 
¥Y ■ KT*i 

GO TO 60 

ERIC 



VARI?12 

VARIM2 

VARIM2 

VARIf<2 

VARIM2 

VARIrf2 

VARIM2 

VARIM2 

VARIM2 

VARIM2 

VARIM2 

VARIM2 

(/ARIM2 

VARIM2 

VARIH2 

VARIM2 

VARIH2 

VARIM2 

VARIM2 

VARIM2 

VARIM2 

VARIM2 

VARIM2 

VARIM2 

VARIM2 

VARIf<2 

VARIN2 

\/ARI>12 

\/ARIM2 

\/ARIM2 

VARIH2 

VARIM2 

S/ARIM2 

VAKIM2 



170 KX ■ KX-H 

IF (KX.NE.f^CQLS) 60 TC 50 
IF (K2CT.NE.»<R0T ) 6U TJ 40 
60 TO 205 

C 

C hUH AND OEM ECOAL 2E»iO 

C 

180 KZCT » KZCT+1 

60 TO 160 
190 IF (TDEN.GE.O) 5L TO 180 

COSPHI ■ 0.707107 

SINPHI ■ CQSPHI 

GJ TQ 140 

C 

C OEn / MUH LESS THAN E 

C 

200 C0S4T • 0.0 
SIN4T • 1.0 
GO TO 110 

C 

C OUTPUT FACTORS 

C 

205 00 220 J - l,NCnLS 

f<jQT(J) » 0.0 

00 210 i » 1#NkL».S 

A(I , J) = A{ I, J)*H(I) 
210 RCfiT(j) » ROJT(J)*A{ I, j)»»2 
220 VAR{J) - ROCT(J)/FLN 

WRITE {6# 300) (i#I « itNCHi) 

DO 230 I » l,NRCfcS 
230 WRITE (0,310) I,(A(I,j),u . l,NCuLb) 

i^RITE ( 6, 320) (RCijT(J),J » 1,.>,CGL5) 
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VARIM2 
VARIM2 
WARIM2 
VARIM2 
VAKIM2 
VARIM2 
VARIM2 
VARIM2 
VARIH2 
VARIM2 
VARIM2 
VARIM2 
VARIMZ 
VARIM2 
V4RIM2 
VARIM2 
\/ARI*i2 
\/ARI»12 
VAKIM2 
(/ARIM2 
VARIM2 
VARI^•2 
VARIM2 
VARIM2 
\/ARIK2 
VARIM2 
\/ARIM2 
VARI*«2 
,/ARIM2 
^^Rlf^ 
yARIM2 
VARIM2 



c 
c 
c 



WRITE ( 6,330) IVAR(J),J - 1,NC0LS) 
CUHV(I) - VAR(I) 
00 240 I ■ 2#NCQLS 
240 CUHVd) - VAR(I)+CUf1V( I-l) 

WRITE (6»340) (CU«y/(J),J - I,MCOLS) 
IF (NVPCH.LE.O) RETURN 
00 250 I ■ I^NROkS 
250 PUNCH 350, I, ( A ( I , J ) , nCOLS) 
RETURN 

FORMAT STATEMENTS 

260 FORMAT <21H-THE SUM 3F THE FIRST, I3,9H RQOTS IS,F14.4///) 
270 FORMAT <26H0 KQ, C31MUN AL IT IE S, / / ) 

280 FORMAT IIH ,I6,FI4.4) 

290 FORMAT <28H0THE SUM OF COMf-UNALITIES IS,F14.4) 

300 FORMAT(*I y/AR*, 13X,*VARnAX FACTOR L QAO INGS* / 3X, ♦NO . I 8, 9 I II ) 
310 FORMAT (1X,I5,3X,10F11.3) 
320 FORMAT (1H-,8HVARIANCE,10F11.3/) 
330 FORMAT (1X,8HPCT VaR ,10F11.3/) 
340 FORMAT (1X,&HCUM PCT ,10F11.3) 
350 F0RMAT<I7,9F8.5/(10F8.5)) 
ENO 



VARIM2 
VARIM2 
(/ARIM2 
VARIM2 
y/ARIM2 
VARIK2 
\/ARIM2 
(/ARIM2 
(/ARIM2 
VARI?<2 
VARIM2 
\/ARIM2 
VARIM2 
1/ARIM2 
t/AKlM2 
VARI»12 
\/ARIM2 
>/ARIM2 
VARI'^2 
i/AR IM2 
VARIM2 
VARIM2 
\^ARIM2 



C 

c 
c 

c 

c 
c 



SUBROUTINE HOW ( LP, NM, M, R, E , V ,A ,8 ,W1 , W2 ) 

OIMENSION R(l), E(l), y/(l), A(l), B(l), tal{l), W2<1) 

IF (LP-1) 70,60,10 

TRI-0IA6CNALIZE MATRIX 

10 CALL T<?.IOI (LP,NM,R,A,8,W1,W2) 

FIND EIGEN VALUES 



HOW 
HOW 
HOi* 

HQ« 
ri'"lw 
HOW 



CALL EIGVAL ( LP« NH« M «E > A« 3« « I«W2 ) HOa 

If CM.EQ.O) GO TU 30 HOm 

c h:w 

C FIND EIGEf* VECTORS H3w 

C HO* 

K ■ lABS(M) „ HQw 

J • 1 HOW 

DO 20 I « 1#K HOh 

CALL EIGy/EC ( LF» NH, A« 6» E ( I ) A ( J m 1» W2 ) H3w 

20 J ■ J+NM HO* 

C HOW 

C ftESTORE INPUT MATRIX HjW 

C HO'* 

30 NHl ■ N?1+l Haw 

■ NHl H'J* 

LP2 ■ LP*Ni Hlv. 

00 50 I ■ 2#LP2»NK1 HO* 

K » I HO* 

00 '♦O J ■ JJ#LP2#NM HO* 

R(K) ■ K(J) HJ* 

40 K • K*l , HO* 

50 44 ■ JJ-^Niil HOW 

GO TO 70 HO* 

C HO* 

C SOLLTION FOR ORDER 1 HOw 

C HOW 

60 Ed} - R(l} HO* 

V(l) ■ 1.0 riOW 

A(l) - R(l) Hjw 

B(l) ■ 0.0 •» HO* 

70 RETURN \ Huh 

END HO^ 
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SUBROUTINE EIGVAL (LP»NH»M»E»A,B»F»W) EIGVAL 

C EIGVAL 
C EIGEN VALUE SUBROUTINE FOR TRI-OIAGONAL MATRICES Dw^ 1517-UB EIGVAL 

C EIGVAL 

OIHENSIQN E(l)» A(l), B(l), F(l), EIGVAL 

EQUIVALENCE (S1»IS1)> (S2»IS2) EIGVAL 

C EIGVAL 

C FIND UPPER AND LOUER BOUNDS AND NORMALIZE INPU1 EIGVAL 

C EIGVAL 

BO ■ ABS(A(1)) EIGVAL 

00 10 I - 2#LP EIGVAL 

10 BO • ANAX1(B0»ABS(A(I) )+B(I)««2) EIGVAL 

80 - BO^l. EIGVAL 

80R - l./BO EIGVAL 

DO 20 I - 1,IP EIGVAL 

A(I} - A(I)*BOR EIGVAL 

B(II - e(I)*BDR EIGVAL 

- 1. EIGVAL 

20 E(.f - -1. EIGVAL 

DO 150 K ■ 1»LP EIGVAL 
30 IF ((W(K)-E(K))/AMAXl(ABS(W(K))»ABS(E(K)),1.0E-9).LE.1.0E-7) GU TO EIGVAL 

1 150 EIGVAL 

X ■ CW(K)*E(K) )*.5 EIGVAL 

C EIGVAL 

C FIND NUMBER OF EIGEN VALUES »N» GREATER THAN OK EQUAL TG X EIGVAL 

C EIGVAL 

IS2 ■ 1 EIGVAL 

F(l) ■ A(l)-X EIGVAL 

IF {F(l).GE.O) GO TO 40 tIGVAL 

ISI ■ -1 EIGVAL 

N - 0 EIGVAL 

GO TO 50 EIGVAL 



ERIC 
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c 
c 
c 



^0 isi • I 

N ■ 1 

50 00 110 I ■ 2»LP 

IF <6(I).EQ.O) GO TO 70 
IF (B(I-l) .EQ.O) GU TO 80 

IF (AfaS(F(I-l))+ABS(F( I-2)).G6.1.0E-15) GQ TU 60 

f(I-l) ■ F( I-n*1.0E15 

F< 1-2) ■ F(I~2)*1.0E15 
60 F(I) - (A( I)-X)*F(I-1)-B(I )**2*F( 1-2) 

GO TO 90 
70 F<I) ■ (A( I)-X)*SIGr«l(l.#Sl) 

GO TO 90 

80 FU) ■ (A(I)-X)*F(I-1)-SIGN(3(1)**2»S2) 
90 S2 « SI 

IF <F<I).EQ.O) GO TJ IOC 
51 - 6IGN(S1»F( I)) 
IF < IS1.E0.-IS2) GU TO 110 
100 N ■ N+1 
110 CONTINUE 

TRAP EIGEN VALUES IN SMALLER AND SMALLER BLUNiJi> 
N ■ LP-N 

IF <N.LT.K) GO TC 130 

00 120 J > <>N 
120 hi4> - X 
130 N ■ H*l 

IF iLF.LT.N) GO TO 30 

Oa 140 J ■ NJ.LP 

IF (X.LE.E(U)) GO TO 30 
140 E(J) - X 

GO TO 30 



EIGVAL 
tIGVAL 
EIGVAL 
EIGVAL 
EIGVAL 
EIGVAL 
EIGVAL 
EIGVAL 
EIGVAL 
EIGVAL 
cIGVAL 
EIGVAL 
EIGVAL 
EIGi/AL 
cIGvAL 
tIGV^L 
E.IGVAL 
EIGVAL 
EIGVAL 
E IGVAl 
EIGVAL 
EiGVAL 
tIGVAL 
EIGVAL 
EIGVAL 

EIGVAL 
EIGVAL 
EIGVAL 
EIGVAL 
EIGVAL 
E IGVAL 
EIGVAL 




150 CONTIHUE 

C 

C RESTORE INPUT AND ORDER EI6EN VALUES 
C 

00 160 I ■ Ipl? 
A(I) - A(I)«BO 
Bd) - 8(1)^80 
160 FU) - (W(I)4^E(I))«B0*.5 
J ■ LP 
K ■ 1 

00 180 I ■ 1#LP 

IF (A8S(F(K)).6T.ABS(FU)n 60 tO 170 

EU) - FU) 

J ■ J-l 

60 TO 183 
170 Ed) « F(K) 

K • K+1 
180 CONTINUE 

IF nSl6N(I#fl).6E.O) 60 TO 210 

00 190 I • 1,LP 
190 FU) - EU) 

J ■ LP 

00 200 I - 1#LP 

Ed) - F(J) 
200 J - J-l 
210 CONTINUE 

RETURN 

END 
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EI6V/L 
EIGVAl 
EI6VAt. 
EI6VAL 
EI6VAL 
E.:6VAL 
E1'6VAL 
EJi6VAL 
EI6VAL 
EI6VAL 
EI6VAL 
EI6VAL 
EI6VAL 
EI6VAL 
EI6VAL 
EI6VAL 
EI6VAL 
EI6VAL 
EI6VAL 
EI6VAL 
EI6VAL 
EI6VAL 
EI6VAL 
EI6VAL 
t I5VAL 
EI6VAL 
EIGVAL 
EIGVAL 



iUBRflfTINt EIGVEC ( Lf># N"* A# 3# t» V» P» C) bIGVEC 

DlMENilj'i AdI, B(l), Vdl, Pill, Cd) cIGVEC 

^ EIGVtC 

C SET UP SnOLTANfcUUS ECUATICnS FUi< EIGEN w£C10k wiTH El&EN VALtE E EIGWEC 

C EIGVEC 

X « A(l)-E EIGVEC 

Y • B(2) tIGVtC 
LPl - LP-I fclGvEC 
DO ^0 I • l,Lf-l EIGVEC 
IF (AB5{Xi-ABS(8(I+in I 10,20,30 EIGVEC 

10 p( I) « 6(!+l) EIGVEC 

0(11 • A(I+1I-E EIGVEC 

Vdl - B(I*2) tIbVEC 

Z • -X/PCII EIGVEC 

X » Z*0(n*Y EIGVEC 

IF (LPl.EO.I) GC Tn ^0 EIGVEC 

Y - Z*V(I) EIGVEC 
GO TO 40 EIGVEC 

20 IF (X.tU.OI X ■ l.OE-10 tIGvEC 

30 Pd) - X EIGVEC 

iiil) • r tIGVEC 

Vd) - 0. EIGVEC 

X • Ad + l)-(!5d + l) /**y-»-E) tlGV^C 

Y - 8(1*2) tIGVEC 
40 CONTINUE cIGVEC 

C 5IGVFC 
C SOLVE SIfULTANEUL-S EOOATIOnS FCP EIGtS VECTDk Or Ti?I-OI AGJ <AL 'lATKlx EIGVEC 

C EIGVEC 

IF (X.EU.OI Gr T^j 70 EIGVEC 

V(IP) - l./A ei-vEC 

50 I - LFl 5I0VEC 

Vd) « d.-Cd)«V().f'))/Pd) tlGVEC 

X • V(LP)**2+V(I l**2 lIGvEC 
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c 
c 
c 



60 1 ■ I-l 

IF fI.EQ.O) 60 TO 80 

ta> ■ f l.-fQf I)«V(I>l)^y(I)*V(l42) ) )/P(I) 

60 TO 60 
70 VfLP) > I.OEIO 

60 TO 50 
80 X - SQRTfX) 

XR • l./X 

DO 90 I • 1,LP 
90 V(I) • VfI)*XR 

TRANSFORM EIGEN VECTOR TO S3LUTI0N OF ORIGINAL MATRIX 

K • LP 

60 TO 130 
100 K • K-1 
4 • J-MH 
Y • 0.0 

DO 110 I > K>L? 
L • 4*1 
110 Y • Y*V(I)*R(L» 
DO 120 I " K»LP 

I. • J-H 
120 Vdl ■ VCn-Y*R(L) 
130 IF (J.NE.O) 60 TO 100 

RETURN 

END 



EIGVEC 
EIGWEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
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SUBROUTINE TRIOI (L P»Nrt# R# A# B# H# 4i ) T-^IDI 

C fi^IDI 

C TRI-0lA6Uf*ALlZATIuM SUBROUTINE 1517-J3 IRIDI 

C RETIRM ORIGNAL K IN JPPEP TRIANGULAR HALF INCLUDING DIAGONAL TRIOI 

C RETURN MODIFIED ^ MATRICES IN LO*fc.i HALF OF R MAlRIX TRIDI 

C RETLRN NEW DIAGONAL IN A TRIDI 

C RETtRN NEW FIRST OFF DIAGONAL IN B TRIDI 

C TRIDI 

DIMENSION R(l), A(l), 8(1), vid), IRIDI 

LPl - LP-1 HIDI 

LP2 » LP1*M.M*LP TRIDI 

LPP » LP2->lrt TRIDI 

• Ni*l TRIDI 

C TRIDI 

C STCRE QRIGI^JAL DIAGONAL T.-<IDI 

C IRIOI 

1*0 TRIOI 

DC 10 I • i*LP2»NM r-^ni 

• L*l TRIDI 

10 A(L) • R(i) TRiOI 

6(1) « 0. TRIOI 

IF (LP-2) 130»120,20 1^:101 

20 UK • 0 TRIDI 

00 100 K • 2»L?1 IRiDI 

KL » KK*K TRIOI 

KU « KK*LO rSIOI 

kj » K*l TRIOI 

C T.RIDI 

C CALCULATb AND STOKE 'lUUlfldO CjLJ^S r.ATRix ^ TRIDI 

C TRIDI 

» 0.0 TRIDI 

OO 30 J « KL#kL' TRIDI 
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30 SUM • SU«+R(J)**2 


TRIDI 


5 « SOKilSUn) 


TRIDI 


B(K) « SIGN(Sf *R(KL ) ) 


TRIDI 


S ■ 1 • /s 


TRIDI 


l»(K) ■ SORT(ABS(R(KL) )*S + 1. ) 


TRIDI 


X - SIGN(S/b(K)»K(KL) ) 


TRIDI 


R(KL) > W(K) 


TRIDI 


00 40 I • KJ»LP 


TRIDI 


JJ • I+KK 


TRIDI 


ltd) " X*R(JJ) 


TRIDI 


40 R(JJ) * U(I) 


TRIDI 




TRIDI 


CALCULATE NEW R MATRIX WITH RCW K-1 NOW HAVING ZERuS OFF 


2N0 DIAGONAL Tf^'OI 




TRiDI 


00 60 J • K»LP 


TRIDI 


Jw ■ J^l 


TRIDI 




TkIDI 




TR IDI 




TRIDI 




iRIDI 




1 R I Di 


IF (JJ.GT.LP) GD TO 70 


TRIDI 


00 60 I - JJfLP 


TRIDI 


L - L+1 


TRIDI 


60 C(J) - Q( J)^R(L)*W(I) 


TRIDI 


70 X ■ 0.0 


TRIDI 


00 80 J » K»LP 


TRIDI 


80 X - XfW(J)«0(J) 


TRIDI 


X ■ ,5*X 


TRIDI 


DO 90 I - KcLP 


TRIOI 
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90 0(1) » X*»<( I)-C( I) Ti^IDI 

LL ■ KK TRIDI 

KK • KK*'4rt TRIDI 

00 100 I - K»LP IRIDI 

LL • LL+MM FRIDI 

DO 100 J - I#LP TRIDI 

L ■ LL*J TRIDI 

100 R(L) - R(LHQ(I)«W( J)^Q(J)^W(I) TRIDI 

C TRIDI 

C SORT OUTPUT TRIDI 

C TRIDI 

L • 1 TRIDI 

DO 110 I ■ 1»LP TRIDI 

X • A(I) TRIDI 

All) - R(L) TRIDI 

R(L) - X TRIDI 

110 L • L^NMl TRIDI 

120 BILP) - R(LPP) TRIDI 

130 RETURN TRIDI 

END TRIDI 
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APPENDIX D 

EDIT WITH TWO-MODE FACTOR ANALYSIS PART ONE (X2M0FA) 

The HSMS "two-mode" factor analysis program X2M0FA is based 

on the Tucker-Messick procedure for factoring an individual differences 

matrix to obtain idealized subject types. ^ In the HSMS application, 

the subject types are task groupings, and the variables are the scaled 

skills and knowledge categories. HSMS uses the X2M0FA program after it 

has utilized PCVARIM to select the number of principal axis factors to 
2 

rotate. 

The HSMS "two-mode" program is run in two parts to overcome 
the problem presented by the great amount of core storage required for 
the program. The output of "Part One" (X2M0FA) provides card inputs 
for "Part Two" (X2MFA2) . 

3 

X2M0FA is always used by HSMS interfaced with EDIT because 
of the nature of the task data inputs and the need to utilize EDIT to 
provide a logarithmic transformation of the data. 



The program presented in this appendix was developed at the University 
of Illinois. It is a modification of the original Tucker-Messick tech- 
nique. X2M0FA was programmed by Sharon Wolf and Ping Kao under the di- 
rection of L.R. Tucker, H.C. Triandis, and E.E. Davis in the period 
1964 to 1966. It was adpated for HSMS for use with the Control Data 
Corporation (CDC) 6600 computer by Katherlne Kurtz in 1968. Aspects 
of the technique are described in Appendix G of this report, where 
further references are cited. 

The PCVARIM program appears in Appendix C. 

The EDIT program appears in Appendix B. 

D-1 
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DESCRIPTION 

X2M0FA is written in FORTRAN IV, and was used in the Control 
Data Corporation's (CDC) 6600 computer at the Courant Institute of 
Mathematical Sciences of New York University. The operating systems 
in use during HSMS analyses were KRONOS and NOS (Network Operating Sys- 
tem). X2M0FA was stored on magnetic tape in compiled and loaded form 
(i.e., in binary object code in non-relocatable form) and in OLDPL form. 

The HSMS task data are transferred to magnetic tape in the 
form of an OLDPL. A utility program, UPDATE, is used to transfer data 
to tape, correct data, or to generate temporary local files in the 
form of a compile file, which is part of the input file for EDIT. The 
input file for UPDATE is magnetic tape in OLDPL form, and the output 
file is C=DATA. The input file for EDIT is DATA, and the output file 
is TAPE9 (with optional printed output). The input file for X2M0FA 
is TAPE9, and the output file is printed and punched. The punched out- 
put is part of the input file for X2MFA2. 

The tasks and the skill and knowledge variables enter X2M0FA 
and X2MFA2 in the order of the final internal numbering assigned by 
the EDIT program. Therefore, the EDIT "dictionaries" are used to in- 
terpret the two-mode outputs. 

Options 

The two-mode programs allow the user to select whether the 
observation "mode" or the variable *'mode" will be the first to be ro- 
tated. As originally used in Illinois, the observation mode was the 
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'first to be rotated. For HSMS, since we wish the factor structure of 
tasks (observations) to reflect the^ factor structure of variables 
(skills and knowledges), the variable mode is rotated first. (Thus, 
once the number of factors is selected based on PCVARIM analysis, the 
mode one (variable) factors in the X2MFA2 output duplicate those for 
that particular solution in the PCVARIM output.) 

'Because it is possible to confuse language references, the 
following should be borne in mind: 



Rows = observations; program allows for more of 
these than variables. 
Columns = variables; program allows for fewer of these 
than observations. 
Mode 1 = whichever is to be rotated (variables or 

observations) . 
Mode 2 = whichever is to be counter-rotated (obser- 
vations or variables). 



The X2M0FA program permits the user to select a column-by- 
column or row-by~row correlation matrix, a column-by-column or row-by- 
row covariance matrix, or a cross-products matrix. If a cross-products 
matrix is selected, the user can select a floating point constant to 
be subtracted from the raw data. 



HEDLIN: A System Subroutine 

The X2M0FA program which is presented in this appendix calls 
a series of subroutines. Among these, HEDLIN, which is a systems-based 
routine^ has not been included in the listing. It is assumed that the 
user can utilize a comparable routine after reading the description to 
follow. 
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HEDLIN is a routine to label listings by block letters (FOR- 



TRAN callable). It prints a string of up to 50 characters in large 
block letters (10 columns wide and 10 rows high). Five lines are 
printed, ten characters per line. The date and time can be included 
in the message. 

Checks 

HSMS makes an inspection of the output as an error check 

as follows: 



1. Check that the means and standard deviations seem 
appropriate to the data. 

2. Check that none of the printout listings show zeroes 
everywhere . 

3. Check tha^ the diagonals in the correlation matrix 
(if listed) do not exceed 1.00 and are near .99. 



Note: As of September, 1977, the comment cards included in the 
program had not been updated to correct language refer- 
ences to rotation modes and dimension modifications. 
The user should follow the instructions for structuring 
the input file which follows, rather than the comment 
cards in the program file (which are not reproduced 
here) . 



ERLC 
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STRUCTURING THE INPUT FILE: EDIT WITH X2M0FA 



In the set-up presented below, the NOS system in use in 1977 
at the Courant Institute's CDC 6600 is assumed. It is also assumed 
that the reader can refer to the UPDATE Reference Manual or a counter- 
part program. In the following instructions denotes a blank* 



INPUT FILE FOR EDIT WITH X2M0FA 



Name 


Keypunch Cards (one per box) 


Instructions 


Sys- 
tems 
Lards 


.EDIT WITH TWO MODE ONE 


Identification number and 
other information depending 
on system* 






USER and user code* 




CHARGE ( ) 


Charge card* 




T AR17T (Cxi WDl tTCM— T \ 


Identifies OLOPL tape* 




UPDATE (Q,D, 8, C=DATA) 
or 


When selecting tasks using 
*COMPILE form of UPDATE* 




UPDATE ( F , D , o , C=DATA) 


When using all data in OLDPL* 




TTXTT f\KT\fr\J ■mST \ 

UNLUAL) V. ^LUr L 








Identifies program tape* 




COPYBF ( PROGRMS . ED IT ) 


Copies program to local file* 




SKIPF(PR0GRMS,1) 


Skips over PCVARIM prograu on 
tape* 




C0PYBF(PR0GRMS,X2M0FA) 


Copies program to local file* 




UNLOAD (PROGRMS) 






SETTL(IOOO) 


Time limit* 




RFL(200000) 


Defines field length* 




EDIT* 






REWIND (OUTPUT) 


Use only if no printed EDIT 
output is desired. Usually 
not needed at this stage. 




HEADER* EDIT WITH TWO MODE ONE 


If REWIND (OUTPUT) is used, 
this is not needed* Prints 
out heading across a whole 
page* 




RFL( 120000) 






X2M0FA* 






End of Record Card 





ERIC 
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INPUT FILE FOR EDIT WITH X2M0FA (continued) 



Name 


Keypunch Cards (one per box) 


Instructions 


In- 


*COMPILEb 


Calls tasks from OI.DPL. Each 


put 




card starts: ^COMPILE in Cols. 


Cards 


etc. , or 


1-8; Column 9 blank, followed 


for 




by Task Code Numbers separated 


UP 


*COMPILEb , 


by commas; last entry has no 


DATE 




comma. Tasks can be in any or- 




(option) 


der. Consecutive Code Numbers 
can be called as last entry on 
card after comma or as only 
entry on card by writing first 
Code Number, then a period, 
the last Code Numbor of the 
series. A new card must fol- 
low. 




End of Record Card. 


Needed even if no UPDATE cards. 


In- 


TITLEb b b b b etc. 


Columns 1-5: punch TITLE; 


put 




Colums 11-60: punch title of 


Cards 




program, date, and any other 


for 




special identification of this 


EDIT 




run. 




NOLI ST 


If listing of data and some er- 




(option) 


ror checks are to be suppressed. 
Usually selected with X2M0FA. 




DELETEb b b b 


One card for each individual 
skill or knowledge category to 




(option) 


be deleted from input data (if 
not to be deleted by CUTOFF). 
Columns 1-6: punch DELETE; 
Columns 11-18: punch 8-char- 
acter skill code or 8-digit 
knowledge category code number. 




CUTOFFb b b b 


To eliminate skill or knowledge 
categories at or below a selec- 




(option) 


ted frequency from input data. 
Columns 1-6: punch CUTOFF; 
Columns 11-12: punch selected 
frequency (usually 4 or more) 
left justified in field. 




NORMALIZE 


Usually used when EDIT is used 




(option) 


with X2M0FA. 




End of Record Card 





Note: In the following pages options appear which, if selected, coulrl 
produce results different than those obtained using the options 
selected by HSMS, In such cases the IISMS choice is indicated. 



% 
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INPUT FILE FOR EDIT WITH X2M0FA (continued) 



Name 



Keypunch Cards (one pgr box) 



Instructions 



In- 
put 
Cards 
for 
X2 

>K)FA 

Para- 
Meter 
Card 



etc. 



Cols, 1-3 



Cols, 4-6 



Cols . 8-9 



Col. 10 



(ot»tion) 



Col. 11 



Col. 12 



Cols. 13-19 



Cols. 61-67 T A P E 9 



Title Card. Cols. 1-40 read as 
one line. Cols. 41-80 read as 
second line. Punch title of pro- 
gran, date, and any other special 
identification of this run. 



End of File Card 



Number of observations for rows 
of input matrix. (For HSMS this 
is tasks treated as Mode 2.) Up 
to 700. Right justify in field. 

Number of variables for columns 
of input matrix. (For HSMS this 
is skills and knowledge categor- 
ies treated as Mode 1.) Up to 
144. (Redimensioned to 150 but 
never used.) Right justify. 

Number of factors to be extrac- 
ted, up to 12. (For HSMS this 
is the number of principal axis 
factors chosen using PCVARIM 
output.) Right justify. 

Punch 1 for column-by- column 
correlation natrix 
(always used by HSMS). 

Punch 2 for column-by-column 
covariance matrix. 

Punch 3 for corss-products 

matrix. See Column 13. 

Punch 4 for row-by-row correla- 
tion matrix. 

Punch 5 for row-by-row covari- 
ance matrix. 

Punch i for printout of matrix. 
Punch 0 to suppress printout. 

Punch 1 for punched output 

(always used by HSMS). 

If 3 in Column 10: punch float- 
ing point constant to be sub- 
tracted from raw data. (Right 
justify.) 

Name of alternate input file 
when EDIT is used (left justify). 
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t!PT-l FIN 4.6*426 

Two-Hooe P^Rt ok)E 

0VERLAYU2i0FA,0»0) 

PROGRA^I X2M0FA(TAPE9, iNPuT, UUTPU T»PIJNC H, TAPd 1, T A?E 2»TA»'E 3, 
♦ TAP£7"INPUT»TAPE6"JUTPUn 

DIMENSION Ff1T(40)»T{12»700/ »Y'( 150#150)»2(150, 12 ) » R 1( 150) ,A ( 1 50 ) » 

B(150),«RK1( 150)»KaK2(150) 
COMMON // FHT»Y»Z»R1»A,B,<<R<1,WRK2#LAST 
EQUIVALENCE (T»Y) 

MAX NiJMBER OF COLUMNS IN MATRIX 
DATA HAXC / 150 / 

ZERO BLANK COMHUN REGION 



10 



FHTIl I 
FMT<2) 
FHTHI 



lOH START 
lOH X2M0FA 
lOH PART 1 



CALL HEOLINiFMT) 
teRITF(6»l) 
FORMAiaHl) 
LF ■ LOCFIFMTI 
LL - LOCFILASTI 
I ■ 0 

00 10 L-LF»LL 

I- 1*1 
FMT(I) « 0 
IN « 7 

II- 40 

R£AO( IN» 530)NI»Nj,NRGT»NSrD#N 'RT» IPCH, SL'ST* INTAP, ILFK 
WRITE (6»531) NI,NJ,NR0T#N»Sr0yHP.<T»IPCH,SO6T# iNTAO,ILFN 



77/09/15 



KRONOS 
CRONOS 
<RUNOS 

11/19/76 

)(2rtOFA 

<2H0FA 

X2M0FA 

KZMCFA 

X2HCFA 

X2M0FA 

X2M0FA 

X2MCFA 

HQWNCWA 

HOW NOW A 

HDmNO«A 

HJWNOWA 

HlMNOrfA 

HOWNO^A 

H3WN0WA 

HOtlKDWA 

HOMNOtirfA 

HQmNOwA 

HOniNOWA 

X2'iGFA 

KRONCS 

^RG^as 

KRGKD? 
HOikNOWA 
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531 FQAHAT(1Xj3I3j3I1jF7«0j I2jT61jA7) 




n J ii ri Lm A 


FNI « FLDATfMIl 




X 2n t r A 


FNJ « FLQATfNJt 




w '7 ^ n c A 


SFNI • SQRT(FNI) 




\ C rUr A 


SFNJ • SQRTfFNJ) 




y 9 M r p A 
A c n L r ft 


IF (ILFN.EQ.TH 1 GD Tn 






IN ' 9 




o fi kt n c 
t> K U N U > 


THIS VALUE IS SET FOR THE LENGTH OF 


FORMATS CRFAIPn RY PHTTPA 


nu A 
nj wiMJH A 


II • 3 




n J lifnUw A 


19 REAO( IN# 540) (FMT( I) J I»1j II) 




1/ D nw nc 
»\ KUNu^ 


yRITE (6*5411 1 FMTI T 1 • T«l • T T 1 




HltilNuW A 


541 F0RNA7( IXjSAIO) 




_j J u r» ij A 


REWIND 3 




A c nUr A 


60 TO ( 30j20j40j50»60) # NSTD 




V 7 Ml n C A 
A cnurA 


20 URITE (6j650) 




V o a4 n c A 
A Z nur A 


£□ TO 180 




w o ij ri c A 
AZHOF A 


30 bRITE (6#670) 




^ 9 ^ Ti C A 

Ad nUr A 


GO TO 180 




^ 1 ii C A 

AZnorA 


40 bRITE (6j680) SUBT 




A d ^ w. r A 


60 TO 80 




V o M n c A 
A Z n U r A 


50 bRITE (6j690) 




V o M r: C A 
A c. "U r A 


60 TO 7 0 




w 1 ^ r% r* A 

A2iCr A 


60 URITE (6«700) 




w 1 Ai f* A 

X 2 n I r A 


70 WRITE (6#690) 




^ 7 M 1P A 
A c «~ J r ft 


80 CONTINUE 




HDWNO^'A 


IF UNTAP.NE.O) IN - IN1AP 




H 3wNnWA 


DO 170 I » 1#NI 




Hli^^Ow A 


WRKIU) - 0.0 




H Dv^NO'nA 


WRK2(I) - 0.0 




H 3w KQ^A 


90 READ(IN*FHT) (A( J), J-1,NJ) 






100 DO 110 J > 1#NJ 




X2iCFA 


WRK2(1) » ^^Ri'ECD+AC J)**2 




x2i:»rA 
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110 MRKI (1) > WRKK 1) * A( J) 

WRKKl) « .<RK1(1) / FNJ 

WRK2<1) . hR<if(l)-FNJ*'(wRKl{ 1)**2) 

teRK2(l) - S0RT(WRK2(1I)/SFNJ 

WRITE (6»550) I»hRKHl)>wRK2{I) 

DO 150 4 ■ 1»NJ 

IF ( NSTO-<t) 120, 130, 140 
120 A(J) » A{^)-SUBT 

GO TC 150 

130 A(J) . (A( J)-WRKl(l))/(5FNJ*»rRK2(l)) 

GO TO 150 
140 A<J) . (A( J)-WRK l(l))/iFNj 

150 CONTINUE 

00 160 J « 1,^J 

DO 160 K « 1,NJ 
160 Y(K,J) » y{K, J)+A{ J)*A{K) 

C 

170 WRUc (3) (A(j),J > l,Nd) 
GO TO 310 

C 

C KiTO • 1 i.R 2 

C 

180 REirflKD 2 

DO 200 I » 1,M 

kEAO( 1.>4,F^«T) ( A{ J ) ,J-l,Uj) 

CO 190 J « 1,KJ 

WRK2<g) » Wi<n2( J)*A( J)**2 
190 l»RKl(j) » wRKl(J)*A(J)/Fn 
200 WRITE <2) (A(J),J » l,NJ) 

C 

C ^0w HArfb X ''ATRIX "'N TA^-E 2 
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X2M?r-i 

x2'*:fa 

XZIC.'A 

<2ML->A 

X2MCFA 

X2HCFA 

X2«lCFA 

X2MCFA 

X2hOFA 

X2MGFA 

X2nCiFA 

X2M0FA 

X21CFA 

X2*<:FA 

XZ.IC'FA 

XZ-^CFA 

HukNOwA 

XZICFA 

X2-«f'FA 

X21LFA 

X2M'jFa 

XZf.LFA 

<2fr.'FA 
X2S(>FA 
<2fGFA 
Xa^CFA 
X2hC FA 
<2''LFA 



t 



WIXE (6^660 
00 2X0 4 ' iiH4 

WRX2CJI « WRK2Uj-FNi*?URKl(JS)**2) 

210 WRITE (6*550) J# 5»RK1 ( J J > kRK2 ? J ) 
REWIND 2 
DO 280 I • 1«NI 
READ (2) (A(J)^J X 1,NJ) 
IF (NSTD-2) 2oJ*2^0,520 
220 00 230 4 ■ 1>H4 

230 AiJI ■ (AiJ)~WRKl{J>)/{SFNI'»WRK2{ J)) 

60 TO 260 
240 DO 250 J • 1,NJ 
250 A(J} . (A(JI-WRK1(J))/$FHI 
260 00 270 J ■ 1,H4 
DO 270 K ■ 1,H4 
270 Y(K#J) • Y{KyJ)+A(J)*A{K) 
280 WRITE (3) {A{J),J . 1,NJ) 

NOW HAVE Xi«X IN T (GOVS, CORR, OR XPkODS) AND STO VARS CN ]APE 3 

IF (NPRT.LE.O) GO TO 310 

L2 ■ 0 
290 LI ■ L2+1 

L2 ■ HIN0{L2+16,NJ) 

WRITE (6,570) (IpI » L1,L2) 

00 300 I • 1,H4 
300 WRITE (6, 550) I, (Yd, J), J • LI,L2) 

IF (L2.LT.NJ) 60 TO 290 

C 

c 

310 CALL HQH( NJ, MAXC, NR OT, Y, Rl, 2, A, B , WRK I, fc(R<2 ) 

C 

c 
c 
c 

ERIC 



c 
c 
c 



EI6EN (SECTORS NOW IN Z(M BY NROT), ANJ RJuFS A«E iN f<l(N»<UT) 

NOW CALCULATE MATRIX OF FACTOR SCCRES, Z* ♦ X 

THE FACTOR SCORES WILL END UP AS FACTCRS IN T(NKPT» MI) 



X2MGFA 
X2K0FA 
){2MCFA 
XZ.iUFA 
X2rtQFA 
>;2MUFA 
A 2 MO FA 

\2«CFA 

xa'ILFA 
XdrtCf & 

X2.MUFA 
<2«CF/. 
X2MCf A 
X2«0FA 
X2«CFA 
X2*<CFA 
X2MC'FA 
X2nOFA 
X2iCFA 
X2MC=A 
<2M0FA 
X2rtOFA 
X2MCFA 
X2MnFA 
*2?1tFA 
)(2K0FA 
X2M0FA 
X2'^DFA 
<21CFA 
■x2i1CFA 
)^2^GFA 
X21GFA 
<2M(:fa 
KaM'-FA 
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c 

c 
c 



c 
c 
c 
c 



REMIND 3 

00 320 I » l#HftOT 

00 320 J » I, hi 

320 T(I#J) • 0. 

00 330 4 « 

READ <3) ^^^44)$4<l « i»Ka> 

DC 330 I • l^HROT 
DO 330 K • 1,NJ 
330 7(I#J) ■ HK,li*MK)*Tll,4} 

hC« CLTPJT RCClZf PERCENT VARIANCE* ETC. 

CUNVAK « 0. 
TtjTV*.-< « 0. 
DO 3 0 J » 1#NJ 
i'tC TGTVAR « 1CTVAR+K1( J» 
>»RITE (6,580) TCTVAR 
fci^ITt (6,560) 
CO 350 J • 

FCr « (kK J)/TGTVAS»*1GG. 
CU'*V«i « Cj«:\rARtpCT 
350 R-^ira (oi~vO; J':<iio>.PC r,CJ«^Ai< 

Pf<.<N? Cria.<4CrtRi-T:'. v^CfjKi fUK -^lOE 1 

WRi~t V6,f;=tj) 'tj 
».RIfE .:6.f)70J (^,^ « l.NSjTj 
CO 360 J » l»Nj 
360 ^lilTE (0,^sTj j,(-.j,(<). . i.^K!rJ 



NOW CLMPJ.'r. rA FACTjRi) F lir^ ^?DE 1 



<2rtOFA 
X21CFA 
X2I10FA 
K2f»uFA 
X2*<0FA 
X2«CIFA 
X2Mf.FA 
X2fiOFA 
X2nCFA 
X2MUFA 
X21QFA 
<2«0FA 
K2i1tFA 
X2M0FA 
X210FA 
X2.-<GFA 
X2MUFA 
X2NnFA 
X2^CFA 
X2nOFA 
<2»»r:FA 
X2MC;FA 
X2M3FA 
<2rOFA 
<2Mi.FA 
X2<iGFA 

x2-^r:FA 

X2M,.FA 
<2MGFA 
<2^.:.FA 
x2^'JFA 

<2*..Fa 



2j.S 
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OQ 390 K ■ 1»NR0T 

IF { RKK) .GT..001) GC TC 370 

Rl(K) * 0. 

GO TO 360 
370 RICK) » S0RT(R1(K)J 
360 DO 390 4 ■ IpH4 
390 ZCJ»K) » Z{J,K)*R1(K) 

ImRITE (6»600) 

WRITE C6»570) (K#K ■ 1,NR0T) 
00 400 J - i»Nj 
'»00 WRITE <6#550) J, (Z(J,K)»K - 1,NR0T) 
IF (IPCH.LE.O) GO TO 420 

C 

C PUNCH MODE ONE 

C 

L • 1 

LFT2 • HIN0(12»NRQT) 
00 410 J > 1,NJ 
410 PUNCH 620, J» L, ( Z ( J» K ) y K « 1»LFT2) 

C 

C 1/6/70 
C 

C NOW PRINT PA FACTORS FOR MODE Tn/O 

C 

420 WRITE (6#600) NI 

WRITE {6»570) (K»K « 1,NSCT> 

DO 430 I ■ 1#M 
430 WRITE (65.550) I,(T(K»I),K • 1»NR0T) 

IF (IPCH.LE.O) GO TO 450 

C 

C PUNCH rtUOE Twr 
C 



X2hOFA 
X2«aFA 
X2MCFA 
X2MrFA 
X2MCFA 
X2M0FA 
X2nCFA 
X210FA 
X2rtCFA 
X2KCFA 
X2M0FA 
X2HQFA 
X2M0FA 
X2MGFA 
X2M0FA 
X2M0FA 
X2MGFA 
X2f«0FA 
XZrtJFA 
X2W0FA 

iiSi$$$S$$$ )(2M0FA 

X21CFA 
X2«jFA 
X2MCFA 

X2*1CFA 
X2.1uFA 
X21CFA 
<2K,0FA 
X2M0FA 
X2M0FA 
X2MGFA 
X2MtFA 
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t ■ 2 X2MDFA 

LFT2 " MlM0tl2,NR0T) X2hC'FA 

00 <»^0 I « 1»NI <2J10FA 

<.<iO PUNCH 620» I#l>(T(K» I)»< • 1»LFT2) X2fCFA 

C X2M0FA 

C f'RiNT CHARACTERISTIC VECTORS FGK rtC'O E 2 X2M0FA 

C X2MCFA 

450 WRITE (6,6<i0) NI X2MCFA 

00 480 K » 1,NRCT X2I10FA 

IF (R1(K).GT..001) 60 TC 460 X2MCFA 

Rl(K) ■ 0. X2MCFA 

GO Tu 470 X2MCFA 

460 RKK) » l.O/RKK) X2^•CFA 

470 00 480 I » 1»NI X21CFA 

480 T(K»I) ■ T(K»I)*R1(<) x2f.CFA 

WRITE (6',570) (»<»K » 1»NR31) <2rtpFA 

00 49G I » 1»M X2fGFA 

490 WRITE <6»550) I»(T(<»I)»K » 1»NR'JT) X21CFA 

C X2MCFA 

C NG«< CALCJLAlt AND PKIM C3RE "lATKlX X2f.CFA 

C X2KCFA 

WRITE ( 6»710» X2f,CFA 

DO 500 I «= l»^Rv-^ <2"1CFA 

500 WRITE (6»6I0) I,RitI) X2'1CFA 

IF (IPCH.LE.O) G: TiJ 520 X2^LFA 

C X2MrFA 

C PUNCH CURE MATklX <2M0FA 

C XclcFA 

DC 510 I » I»NRur K2MCFA 

510 PUNCH 630» I»kl(I) X2''.CFA 

C X2MurA 

520 CONTINUE X2KCFA 

C )k 2 r. f F A 
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C FORMAT STATEMENTS X2«DFA 

c xanoFA 

530 FORHAT(3I3»3I1>F7.0»I2#161»A7) X2HCFA 

5*0 FORMAT{8A10) KR0N0S2 

550 FORMAT (IH »I3»16F8.3) X2N0FA 

560 FORMAT { 58H CHARACTERISTIC ROOTS PERCENT VARIANCE CUM. PERC X2H0FA 

lEMT) X2M0FA 

570 FORMAT (/////3H ,1618/) FIXFMT 

580 FORMAT (/////18H TOTAL VARIANCE • »F12.3/) FIXFMT 

590 FORMAT (IH I3,5X« F12. 3,6X«F12.2«lX»F12.2 ) X2H0FA 
600 F0RMAT{/////27H PRINCIPAL AXES FACTORS FOR* I<»# 1<»H-VAR lABLE HOOE) FIXFMT 

610 FORMAT (IH I3»F12.3) X2I10FA 

620 FORMAT { I3# II, 12F6.2 ) X2M0FA 

630 FORMAT {I3#F12.7) X2M0FA 

6*0 FORMAT{ /////**H CHARACTERISTIC VECTORS (FACTOR SCORES) FOR # I*# FIXFMT 

P 1*H-VARIABLE MODE) FIXFMT 

650 F0RMAT(/////30H COLUMN BY COLUMN COVARIANCtS ////) FIXFMT 

660 FORMAT (31HiMEANS AND STANDARD DEVIATIONS ///) X2M0FA 

670 F0RMAT(/////3OH COLUMN BY COLUMN CORRELATIONS ////) FIXFMT 

680 F0RMAT(/////16H CROSS PRODUCTS .» F 8. 0, 33H «Ai SUBTRACTED FRGM EACH FIXFMT 

♦ENTRY. ////) FIXFMT 

690 F0RMAT(/////2*H KOW BY ROW CORRELATIONS ///) FIXFMT 

700 F0RMA1(/////23H ROW BY ROW COVARIANCES ////) FIXFMT 
710 F0RMA1(/////36H CLRE MATRIX* EUUALS 1. 0/ SORT (i<OCT ) # /^bH SO IS CO FIXFMT 

♦RE MATRIX FOR THE PA FACT3RS OF THE TwQ MOOES. ///) FIXFMT 

END X2M0FA 
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SUBROUTINE HOW ( L«>, N!4, R,E » V » A, B, « 1, w2) 
0I»«ENSIJN Rd), Ed), V(l), A{1), 8(1), wl(l), W2(l) 
IF <LP-1) 70,60, 10 

C 

C TRI-0IAG0NALI2E MATRIX 
C 



HG* 
HOW 
HjW 
HQw 
HOw 
HQw 
H'lW 
HQ«« 
HIW 
H34 
H3m 

C FIND EIGEN VECTORS H^*. 



10 CALL TRIOI (LP,NH,R, A,3,wl,w2) 

C 

C FIND EIGEN VALUES 
C 



CALL tIGVAL ( LP,NH,M,E,A,8,»,l,w2 ) 
IF (M.Ew.O) GO TO 30 



H'JW 

CALL EIviVEC (LP,NM,R,A,fa,t(I),v( J),Wl,,-2) h'ih 



K • IA8S<^) 
J • 1 

00 20 I ■ 1,K 



20 J ■ J + N»* 

C 

C RESTORE iNP'Jf MATRIX 
C 

30 NMl • Ni+1 
44 » NMl 
LP2 • LP»N^ 
00 50 I ■ 2,LP2,Nri 
K - I 

DG 40 J » JJ,LP2,^M 
RCO • R(J) 
40 K • i^ + l 



H 3» 
HOW 

H jh 

H'J-. 




50 44 ■ 4J*K«1 HQW 

60 TO 70 HOW 

C HOW 

C SOLITION FOR QROER 1 HOW 

C H'DW 

60 Ed) - R(l) HOW 

VU) ■ 1.0 HOW 

Ad) - R(l) HQW 

8(1) - 0.0 HOW 

70 RETURN HOW 

END HOW 



SUBROUTINE EIGVAL ( LP# NH#«1,E # A, B# W ) EIGVAL 

C EIGVAL 

C EIGEM VALUE SUBROUTINE FOR TRI-OIAGONAL MATRICES DWH 1517-UB EIGVAL 

C EIGVAL 

DIMENSION E(l)# Ad)« B(l)» F(l)» Wd) EIGVAL 

EQUIVALENCE (S1#IS1)# (S2,IS2) EIGVAL 

C EIGVAL 

C FIND UPPE^ AND LOwEk BOUNDS AND NDkMALIZE INPUT EIGVAL 

C EIGVAL 

BD - ABS(Ad)) EIGVAL 

DO 10 I ■ 2,Lf ciGvAL 

10 BD « AMAXl(BD#ABS(A(I))*3d)««2) EIGVAL 

3D ■ BO-H. EIGVAL 

8DR « l./BD EIGVAL 

DO 20 I ■ 1,LP EIGVAL 

Ad) ■ Ad)*BDR EIGVAL 

Bd) ■ 3d)*80R EIGVAL 

Wd) « 1. EIGVAL 

20 E( 1) ■ -1. EIGVAL 

DO 150 K » ItLf' ciGVAL 
30 IF <<W(K)-E(K))/AMAXl(AB5(k(K)),A8S(E(K))#1.0t-<>) .LE.l.OE-7) G3 m EIGVAL 

1 150 EIGVAL 

O X ■ <W(K)*£<K) )».5 EIGVAL 



c 
c 
c 



c 
c 
c 



FIND nmbtH C'f EIGEN VALuti ,N, GREAfLR THAN cwLAL TC X 

1S2 • 1 
F(l) « A(l)-X 
IF <F(I).GE.O) GU 13 **0 
ISI • -I 
N • 0 
GO TO 50 
<»0 ISI - 1 
^ « I 

50 DO 110 I - 2»LP 

IF (8{I).Et.O) Gci TO 70 
IF (8(1-1). ECO) GO TU 8j 

IF (AeS(F(I-l))+ABS(f ( I-2)).Gf .l.OE-lt)) bj 10 60 

F(I-l) . F( I-l)»1.0ti:> 

F(I-2) « F{I-2)*1.0615 
60 Fd) • (A(I)-X)*F(I-1)-3(I)**2*F(I-?; 

G3 TO 90 
70 Fd) « (A(I)-x)4SIoN{l.,il) 

GG T3 90 

60 F(I) - ( A( I)-X )*F(I-1)-SIJN( 3(1) ♦♦2» S2 ) 
90 £2 • SI 

IF ( F( I) .EJ.O) GO Tl IOC 
SI « SIG.i(Sl,KI ) ) 
IF (Iil.Ew.-lS2) G'., rC 110 
100 N • N+1 
110 CONTINUE 

Tt^AF EIGEn y/ALUES IN SMALLER AmD Skall^R iUUNuS 
N • LP-N 

IF (N.LF.K) GD TC 130 

DC 120 J ' 



= loVAL 
tIGVAL 
E IGVAL 
EIGVAL 
E IGVAL 
E IGVAL 
EIGVAL 
EIGVAL 
EIGVAL 
EIGVAL 
E IGVAL 
E IGVAL 
tIGVAL 
d IGVAL 
E IGVAL 
EIGVAL 
tIGVAL 
t IGVAL 
EIGVAL 
tIGVAL 
£ I3VAL 
tIGVAL 
E IGVAL 
E IGVAL 
t IGvAL 
EIGVAL 
EIGvAL 
EIGVAL 
E IGVAL 
E IG vAL 
EIGVAL 
t IGVAL 
t IGVAL 
£ IGVAL 
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c 
c 
c 



120 ki(J) > X 
130 K ■ N+1 

IF (LP.LT.N) GO TO 30 

00 140 4 • Htl? 

IF (X.LE.E(J)) GO TO 30 
140 E(J) « X 

60 TO 30 
150 CONTINUE 

RESTORE INPUT AND ORDER EIGEN VALUES 



I ■ UiP 

A(I)«BD 
B( n*BD 



DO 160 
A(I) - 

B(n • 

160 f(I} • 
J ■ LP 
K - 1 

00 180 I - 1,LP 

IF (ABS(F(K)).GT.ABS(F( J))) GO TO 170 

E(I) - F(J) 

J • J-1 

GO TO 180 
170 E<II - F(K) 

K - K*l 
180 CONTINUE 

IF (ISIGN(1#«1).G£.0) GO TO 210 

DO 190 I - 1,LP 
190 Fd) - E(I) 

J ■ LP 

DO 200 I » 1,LP 

E(I) - F(j) 
200 J J-1 
210 CONTINUE 

RETURN 

END 



EIGVAL 

EIGVAL 

EIGVAL 

EIGVAL 

EIGVAL 

EIGVAL 

EIGVAL 

EIGVAL 

EIGVAL 

EIGVAL 

EIGVAL 

EIGVAL 

EIGVAL 

EIGVAL 

E IGVAL 

EIGVAL 

EIGVAL 

EIGVAL 

EIGVAL 

EIGVAL 

EIGVAL 

EIGVAL 

E IGVAL 

EIGVAL 

E IGVAL 

EIGVAL 

EIGVAL 

EIGVAL 

EIGVAL 

E IGVAL 

EIGVAL 

EIGVAL 

E IGVAL 

E IGVAL 

EIGVAL 
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c 
c 
c 



c 
c 
c 



SU8RQLTINE EIGVEC ( LP# Nf , R, A , 3# E # V# » # 0 ) 
DIHEnSIJN R(1), A(1), 5(1), V(l), Pll), HI) 

SET UP SMULTANEDUS EgUAlIONS F QK EIGEN VECT.Jk kilH EI6EN VALUE E 

X « A(U-E 

Y - 8(2) 
LPl « LP-1 

DQ <.0 I » 1,LP1 

IF {ABi()()-ABS(e( I + l) ) ) 10,20,30 
10 P(I) » B( Ul) 
CCD • Ad + D-E 
Vd) • 8( 1*2) 
Z • ->/p( I) 
> » 2*j(I)+Y 
IF (LPl.Efi.I) 6G T3 'lO 

GO TO 40 
20 IF (X.e-.O) X » l.Ot-10 
30 P(I) « X 

t(n • r 

V{ i> » 0. 

X « A(I+l)-(d(I+l) /X«Y+£) 

Y « 6(1+2) 
<»0 CG^TI^u£ 

SuLVfc hI^JLTA.^^EC•LS• tiOATIoNS FiP EIGt*^ VcClJP. uF JkI-jIAGJ^ai i^ATkIX 

IF (X.Ej.O) GC TL' 70 
V(LJ>) = l./x 
^0 1 « LPl 

U I) « ( 1 .-.( I j + i^dr- ) ) /P( I) 



tIGVEC 
fIGVEC 
E IGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
E IGVEC 
tIGVEC 
EIGVEC 
EIGVEC 
tIGVEC 
tIGVEC 
EIGVEC 
cIGvEC 
tIGVEC 
EIGVEC 
tIGVEC 
E I6VEC 
E IGVEC 
c IGVEC 
EIGVEC 
t IGVEC 
tIGVcC 
EIGVEC 
EIGVEC 
tl GwEC 
E I GvE C 
c IGViC 
b 13 VEC 
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c 
c 
c 



60 I - I-l 

IF CI.Ea.O) GO TO 60 

V{I> » (l.-(Q(I)*V(I+l)*)/(I)*rf( I*2)))/P(I) 

X - X^V(I)**2 

GO TO 60 
70 VILP) • I.OEIO 

60 TO 50 
80 X - SQRT(X) 

XR • l./X 

00 90 I ■ 1,LP 
90 veil • V(I)*XR 

TRANSFORM EIGEN VECTOR TO SOLUTION OF ORIGINAL MATRIX 

4 • LP1*MM-NM 
K • LP 
GO TO 130 
100 K • K-1 
J • J'-NM 
Y • 0.0 

DO 110 I » K#LP 
L • J+I 
110 Y • Y+V(I)*R(L) 
DO 120 I > K»LP 

L • J*I 
120 V(I) • y/(I)-Y*R(L) 
130 IF (J.NE.O) GC TC 100 

RETURN 

END 



6IGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
E IGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
EIGVEC 
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f 



c 
c 
c 
c 
c 
c 
c 



c 
c 
c 



c 
c 
c 



SuBROtTlNE TRIO I (LP7mi7R#T,B,V.#0) 

TRI-OIAGONALIZATinN SUBRGUTINiE 1517-uB 
RETtRN GRIGNAL R IN UPPER IRIANGULAK HALF INCLUDING DIAGONAL 
RETURN PCJIFIEO W MATRICES H LOWER HALF OF K rtAlklX 
RETURN NEW DIAGONAL IN A 
RETURN NEW FIRST OFF DIAGONAL IN B 

DIMENSION R(I), A(l), 3(1), 0(1), w(I) 

LPl » LP-I 

LP2 » LPI*NN+LP 

LPP • LP2-Nf^ 

Nfl • ({^*\ 

STORE ORIGINAL DIAGJNAL 
L • 0 

DC 10 I - 1,L»'2,.'.'<1 
L » L+1 
10 A(L) > K(I ) 
fc(l) « 0. 

IF (LP-2) 130, 120,20 
20 KK » 0 

DC IOC K » 2, LPl 
KL • K<^+< 

HJ » K+1 

CALCvJLATl and SlCKt idUlFlLL) CuLJMN lAF^K 4 



iU" = 0.0 

C iJ J « <L,i'L 



TRIDI 

TRIDI 

TRIUI 

TRIDI 

TRIOI 

TRIDI 

TRIDI 

TRIDI 

TRIDI 

T-<IDI 

TRIDI 

TRIDI 

TRIDI 

TRIDI 

TRIOI 

T^IDI 

TRIDI 

TRIDI 

TRIDI 

T^IDI 

TRIDI 

TRIDI 

TRIDI 

TkIDI 

TRIDI 

TRIDI 

TRIDI 

TRIDI 

r-^IDI 
r«lDI 
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30 SUN ■ SUn*R(J)**2 
S ■ SQRT(SUM» 
B(K) ■ SIGN(S«-R(KL)) 
S ■ l./S 

W(K» ■ SQRT(ABSiR(KL»»*S*l.) 
X - SI6N(£/W(K)>R(KL) ) 
R(KL) - W(K) 
00 *0 I - KJ#LP 
JJ - I4-KK 
bd) - K*R144) 
40 R(JJ) - U(I) 

C 

C CALCULATE NEW R HATRIX WITH ROW K-1 
C 

00 60 J ■ K»LP 

JJ - J4l 

0(J) - 0.0 

L ■ KK*J 

00 50 I ■ K,J 

L " L+NM 
50 Qi4i - U<J)^R(L)*w(I) 

IF (Jj.GT.LP) GO ID 70 

OG 60 I - JJ,LP 

L - L*l 
60 0(J} - 0( J) ♦R(L)*W(I) 
70 X ■ 0.0 

DO 80 J ■ K,LP 
80 X ■ X*W(J»*0(J) 

X ■ .5*X 

00 90 I ■ K,LP 



TRIDI 
TRIDI 
TRIDI 
TRIDI 
TRIDI 
TRICI 
TRIDI 
TRIDI 
TRIDI 
TRIDI 
TRIDI 
T^IDI 

NOW HAVING ZERDS OFF 2ND DIAGONAL TRIDI 

TRIDI 
TRIDI 
TRIDI 
TRIDI 
TRIDI 
TRIDI 
TRIDI 
T'^IOI 
TRIDI 
T^IDI 
TRIDI 
TRIDI 
TRIDI 
TRIDI 
TRIDI 
TRIDI 
TPICI 
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VO Will » A*W|il*'Vilil 


TR lOl 




IRIOI 




TRIDI 


UQ 100 1 • JSfLr 


TRIDI 


II m II^Ai^ 


TR IDI 


nr 1 AA 1 M T 1 D 
Uu 100 4 * l#Lr 


TRIDI 


1 M t 1 ▲ 1 


TR IDI 


100 KILI * K 1 L 1 1 1 1 C J 1 > Ql U 1 ▼kl II 


TRIDI 




TRIDI 


SuRl UtTPUT 


TRIDT 




TRIDI 


L • 1 


TRIDi 


00 110 1 • I^Lr 


TRIDI 


A • A 1 1 1 


TR IDI 


AITI « Dill 


TRIDI 


RCLI • X 


TRIOI 


110 L • t ♦NMl 


TRIDI 


120 e(LP) • K(LP?> 


TRIDI 


130 RETURN 


TRIDI 


END 


fRIDI 
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APPENDIX E 

rWO-MODE FACTOR ANALYSIS PART TWO (X2MFA2) 

Tht?' .i/>iv. •< presents the second half of the HSMS "two-modo" 
:.^C'ur rro-rain/ Part Two (X2MFA2) utilizes the punched out- 

.^uc f \;\ Part One (X?M0FA; . Thf: description to follow indicates how 
zhe cjrds mcv structured lo become the input file for X2MFA2. 

X2>fFA2 is written in FORTRAN IV and was used in the Control 
r>aca Corporation's (CDC) 6600 computer at the Courant Institute of 
Mathematical Sciences of New York University. The operating systems 
in use during HSMS analyses were KRONOS and NOS (Network Operating 
System). X2MFA2 x^as stored on magnetic tape in compiled and loaded 
form (i.e., in binary object code in non-relocatable form) and in 
OLDPL form. 

The X2MFA2 program permits the user to select whether the ob- 
servation "mode" or the variable "mode" will be the one rotated, and 

2 

which will be counter-rotated. The maximum number of factors is pre- 
selected in X2M0FA, but can be fewer than the number selected at that 
time. 

The tasks anu the skill and knowledge variables enter X2M0FA 
c>.id X2MFA2 in the order of the final internal numbering assigned by 



^ fee Appendix D for a descriotion of Part One (X^MOFA) . 
2 

Mode i wh ichv^.vf^M^ is to hft ror.MieJ vvariabk.s or on^^ervat ions ) . 

Modr-^ ? ' whirhever is m be v*o\itv.er-rot:at ed • ohserv - » -on^ or variables) 
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the HSMS EDIT program. Therefore, the EDIT "dictionaries" are used to 

3 

interpret the two-mode outputs. 
CHECKS 

If the factor structure obtained with the PCVARIM program is 
for the same mode that is rotated in this program, then the first fac- 
tor matrix in the printed output should be equivalent, except for 

rounding errors, to the factor structure obtained with PCVARIM foK the 

4 

given number of factors. For example, HSMS selects a column-by-column 
correlation matrix to obtain the factor structure of variables- Tl:is 
is the mode rotated in X2MFA2; the factor matrix of PC7ARIM for the 
factor solution chosen (number of rotations) approximates the first 
and second factor nntrixes in the X2MrA2 princouc. 

As another che.'k, the first and second factor matrixes should 
be the same except for rounding. 

STRL'C TI " >ING THE INPUT FILE FOR X2> rrA2 

The punched output from Part One '.X2M0FA) muh:t he separated 
and arranged in appropriate order for submission. The o^der for sub- 
mission reflects d.^ choice of modes to be rotated and c ounter-rotatpi . 
This need not correspond to the order in which the punched output aD- 
pears; in the case of HSMS usage, the punched output must be rearranged. 



3 

KDIT is described in Appendix B. 

4 

PCVARIM is described in Appendix C. 
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The piunched output is re.aliy three separate decks. Each deck 
Is Jndi vldualiv numbered In r.j>:urms i through 4. Column 4 is used to 
•idv.ritify all the caraa of th(. same d-ack; i.^., is blank or is punched 
with a 1 or a 2. Colanirs 1 through 3 number the cards for the f^iven 
deck. 

The cards coiae out iv. whfA following ord^-r, and are rearranged 
for submission as indicated in the iastructlons which follow. 

DecK ]: Principal axis factors for the variable mod« 
(HSMS's Moda 1 colun^n mode, the one to be ro- 
tated* As many cards as the number of skill 
and knowledge variables). 

Deck 2: PrxDcipal .axis factors for the observation mode 
(HS?lS*s Mode 2 vow xode, the one to be count er~ 
rotated. As many cards a^; the number of task 
ccservations ) , 

Deck 3: Core Mtrix cards. As many cards as the number 
of factors extracted in X2M0FA. 
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f Kamc- 



INTMJT 'niK FOR X2MFA2 



Ti» ^.tructi'>Fv- 



put 

Cards 

For 

X2M 

FA2 

Para- 
meter 
Card 



CHARGE ( 



LABEL (PROGR>i 



Idenrificatlon nu^-iiber r,)id 
other information dcponding on 

;-:v5>t5vn. 

rSER and user c. de. 



)j Cfiai ge card . 



I T.^'rTitif ies OLDPL (ape 



SKiri'(PR0GR?^S,3) 



COPYBF (PROG RMS,x :2WA2) 
UMLOAD(PRQGRMS) 



1'; kips over EDIT/ PCVARIM and 
X2M0 FA on tape. 



RFL(120000) 



HEADFR.TUO MODE 2 



etc. 



X2MrA2. 



EnJ. of Record Card 



Cols. 1-3 



Cols. 5-6 



Cols. 7-9 



Col. 10 



Col. 11 



Cop ies proji;>ram to local file. 



-Pi.- y.^cs field length . 



Prints out heaoing across a > 
lih '^^le page. \ 



Right justify in all fields. 
Number of units in mode to be 
rotated. For HSMS this is 
skill and knowledge variable 
mode. Up to 700; for HSMS, 
up to 144. This is Mode 1. 

Number of factors to be ro- 
te? ted. Up to che number se- 
l'.h:ted for Pa) c One. For HSMS 
s-ime as factor solution cho-- 
sen. This is the core matrix. 

Number of units in mode ?:o be 
counter-rotated. For HSMS 
this is task mode. Up to ZOO. 
This is Mode 2. 

Punch C for no punched output; 
Punch 1 for punched output. 

Punch 0 to rotate Mode 1 and 
counter-rotate Mode 2 HSMS 
always uses this. 
Punch 1 to rotate each mode 
separately and counter-rotate 
the core matrix. 



ERLC 
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INPUT FILE FOR X2MFA2 (continued) 



Name 


Keypunch Cards (one per box) 


Instructions 


In- 
put 
Cards 
For 


Cols. 1-10 (3X,F12.7) 


Format card for core matrix 
deck. For HSMS, punch as 
shown. (Can be up to 14 
columns . ) 


X2M 
FA2 

\ \. Ull 

tin- 
ued) 

Core 
Ma- 
trix 

Deck 


Insert core matrix deck next. 
As many cards as the number 
of factors to be rotated 
and counter-rotated . 


If the number of factors 
extracted in Part One was 
greater than number desired 
now, use the proper number 
of cards in the order found 
in the punched output. Set 
the remaining cards aside 
for possible later use. 


Mode 
1 

Deck 


Cols. 1-7 (13, IX, 


Format card for Mode 1 deck. 
For HSMS skill and knowl- 
edge variable mode; punch as 
biiuwri . 




Cols. 8-9 


Punch the number of factors 

tify. (Same as cols. 5-6 
in parameter card.) 




Cols. 10-14 F6.2) 


For HSMS punch as shown. 




Insert Mode 1 deck next. 
As many cards as the number 
of units in Mode 1. 


For HSMS this is the skill 
and knowledge variable mode 
deck . 


Mode 
2 

Deck 


Cols. 1-7 (13, IX, 


Format card for Mode 2 deck. 
For HSMS task observation 
mode; punch as shown. 




Cols. 8-9 


Punch the number of factors 
to be counter-rotated. 

rvxgiii. J us I. X 1 y . \Ociiuc do 

cols • 5—6 in parameter 
card.) 




Cols. 10-14 F6.2) 


For HSMS punch as shown. 




Insert Mode 2 deck next. 
As many cards as the number 
of units in Mode 2. 


For HSMS this is the task 
observation mode deck. 




End of File Card 
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♦ 

c 
c 
c 
c 
c 
c 
c 
c 
c 
c 
c 
c 
c 
c 
c 
c 
c 



c 
c 

c 

c 
c 



OVE«LAY{J^2«FA2,0»0) 

F«03RA^ X2"F A2 ( I NPIIT» LUTPJT» fuNCH, TAr ^7- INkuI , T Akl6« jvl -uT 

rART II CF 2-f"('0E FA RlTAri3\ 

J-ARATEt<nElER CAkDS 

1. 13 NRl • NL. LF »AkIA3L£S JF FlkST ItJt F RjTATIJn, 10 
13 NF - Nu. uF FACT3R fj Be R3TAT50 

13 NR2 - OIMENj>IUv» GF GfHER MJDt» TQ 700 

U N^CH » 1 IF *A.ST FUSCH JUTPJT» JTHER^Iit 0 

U in - aPTlCN 0 RCTATE FIRST -^OE AmD CJU'^TER R'.iAl 

3THER MuDE 

1 RCFATE EACH MODE iE PAR A F E L Y »C L UMER 

THE CCRE "ATkIX 

2. FORMAT Fi.R CTRE "ATRlA 

3. CCRE c.ATRIX OAlA {AS lANY AS Nu. '^F FACT-.'R) 
<». F-JRMAT JF Pa factor HkST MODE 

5. OAlA GF PA FACT'J^ bF FIRiT PCDt 
fc. FGRWAT jF pa factor CF CTHER r.jDE 
7. JATA jF PA FACT'R jF JTHlR I")D£ 

CL-^nuN // A( 700,10)»K(10)»H( 700),Fini«,; ,v(10),<:(700, 10) 
L » T{10,10) , TTdUflO) 
REAL wEC(lO) 



PEAJ {7»2n) ,^F,-<R2»nJCh,ln 

V CONTAIN R^.:rS»R C IMIAIm l.O/f' i'. F 



700 



K ]TAT 



READ I7,2^i0) ^^l 



RtAD (7,i-«*T) 
C J IG 1 • 1,M- 
\. (I) = l./R( I)*»2 
VtCd) - 



I » >*>^ ) 



23C 



77/JV/lp 



XZ.MP A2 

cxrt^o 

, aTENO 
X2MF A2 
X2MF A2 
X2rF A2 
<2«F A2 
A2nF A2 
X2.'*»FA2 
X2MF A2 
X2«FA2 
X2MFA2 
X2^F A2 
A2 

yZ^F A2 

?:AIE^O 

X 2HF A2 

X2^FA2 
xZ •^F A2 
\Z>& A2 
X2MF A? 
xZ^r A2 
J»2 ■ F A2 
k2 -FA2 
F A2 
X ^NF A2 
X ^ - P A2 
^ Z H A2 



ERIC 



c 
c 
c 



c 
c 
c 
c 



IS INPCT bET 2 



READ IN FIRST MODE —PA FACTOR 

MODE » 1 
HR • hRl 
20 READ (7#220) FHT 
DO 30 I ■ I»NR 

READ (7,Frtr) KC»( A(I,J) , J . i,HF ) 

IF( KC .NE. I ) CALL ABORK 32L ERROR 

00 30 K • I»NF 
30 S(I#K) • A(I»X) 

00 40 I - 1»NR 

H(I) - 0»0 

00 40 J • l,Ht 
40 h(I ) • H< I)+A( I, J)**2 

»>RITE (6»230) NR 

CALL VARIMX(NR#NF,NPCH) 

F* IN A(700»10), F IM $(700»I0) 

GET F*F* , THEN PREMULTIPLY BY ROCiT**-I 



CALL MSATB ( S»NR» NF# A# NR»SF,T» 7G0» 700» 10) 
WRITE (6»50> 

50 FORMAT (40H-F0R SOME CHECK, THERE ARE THE EION RCHT':) 

DO 60 I » 1,NF 

WRITE (6#2f»0) I,VEC(I) 

DO 60 J • i,NF 
60 T(I»J) » T{ I»J)/VEC( I) 

liRITE <6»240) 

DO 70 I « l^MF 
70 WRITE (6»2i>0) I,(r(I»J),J - 

WRITE (6»80) 



X2'<FA2 
X2MFA2 
X2MFA2 
X2.1FA2 
X21FA2 
X2rFA2 
X2MFA2 
X2I1FA2 
) X21FA2 
X21F A2 
<2ttFA2 
X2MFA2 
X2MFA2 
<2MFA2 
X2MFA2 
X2HFA2 
X2MFA2 
X2HFA2 
EXTEND 
X2iiFA2 
X2MFA2 
EXTEND 
X2MF A2 
X2MFA2 
X2NFA2 
X2rtFA2 
X2HFA2 
X2HFA2 
X2MFA2 
X2HFA2 
X2MFA2 
X2MFA2 
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80 


FCKMAl (16H-F0R CHECK, iS»»«n 


XZrtFAZ 






CALL nSATB (T,NF fNFfT^NFf'tF, A, 10, 10,700) 








00 90 I • 1,,-^F 


X F A? 




90 


taRITc (6,230) I,(A(I,J),J > 1,nF) 


XZ-IF A2 






WRITE (6,100) 






100 


FORnAl (16H-FLR CHECK,FT«F») 








CALL ►SAB (S#^R,hF,T,NF,NC, A,700,10,700) 


V ^ 1 U 






00 110 I • l,Nk 


X2^F A2 




llO 


WRITE (6,230) I,(A(I,J),U » 1,NF) 








IF (LN.6E.1) GO U' 1<»0 


X2liF A2 


c 






X2MF AP 


c 




COUNTER POIATICN OF OTHER hCQE 




c 






FAP 

^ Cm • r t 






READ (7,220) FMl 


X?MFA2 






DC' 120 I • l,Nk2 


X2MF A2 






READ (7,Fr;r) KC, ( A(I,J),J » l#Nr ) 


^ c n r f\ c 






IF( KC .NE» I ) CALL ABCiiT( 33L tRRIR I< KHCT SET 3 


1 t PMP A? 




120 


CONTINUE 


A i n r #• ^ 








)(2nFA2 






CALL ^SaB ( a, NK2,UF, r,NF,NF, S, 700» 10,700) 


t J( TENC 

u ^ 1 t W 






WRITE (6,260) NR2,(II,II « 1,NF) 


X2*1F A2 






DC 130 I • 1,NR2 


<2iF A2 




130 


WRITE (6,230 ) I, (:>(!, J), J - 1,NF) 




c 






K 2MF AP 


c 




ROTATE C3RE MATtilX 


^ Cm t * 1 Cm 


c 






n Cm W M t 




14C 


DO 150 I • 1,MF 


X P AP 






C-Q 150 J » 1,NF 


M 7 MC Ap 




150 


TT(I,J) • T{ I, J) '^i^( J ) 


* P M i- Ap 






IF (LK'.GE.l) GO Tn 170 


X2HF A2 






•rftlft (6,270) 


X2**FA2 






WRITE (6,280) (II, II - l,^r) 


a2 IFAZ 






CALL ►ijAdT (TT,KF,NF, r,:'.Ft'w?,A,<10» 10,700) 


LUfcNO 






160 I " l,Nf 


)C2'-FA2 
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23S 



160 



C 
C 
C 



WRITE 
60 TO 



(6»250) 

200 



I»(A( I»J)»J - 1,NF) 



170 



180 



190 

200 



C 
C 
C 



210 
220 
230 
240 
250 
260 

270 
280 

290 



ROTATE OTHER KGOE 

IF IHODE.NE.I) GG TO 18C 
KR • NR2 
HODE ■ 2 
60 TO 20 
WRITE (6,270) 

WRITE 16,290) (11,11 . l,MF) 

CALL f»SAB (TT,hF,NF, T#Nf ,'^F, A, 10/ 10, 700) 
00 190 I ■ 1,nF 

WRITE (6,250) I,(A(I,J),J « l,NF) 
CONTINUE 

FORMAT STATEMErT 



FORMAT 
FORMAT 
FORMAT 
FuRMAT 
FORMAT 
FORfAT 
111/) 
FORMAT 
FORMAT 
1) 

FOR^'AT 
IRST 
END 



(313,211) 
( 13A6,A2) 

(lril;.llHVARI,vAX FQR,I3,14H VARIABLE t'uOE//) 
( lH-,16HTRASSFC'Rh iATRIX//) 
( IX, I3,10F11.4) 

(lHl,19HCuUNTER RQTATIUN nF,I3,14H VARIABLE Mf 0 E / / 10 X , 1 1 , 9 1 
(lH-,34HRuTATED CORE HATRIK G (NFl bf nF2)/) 

(lX,38HDATA-(F*)G(Fj) , ^HERh. FJ li PA F AC F CR // 1 OX, 1 1 , 9 1 1 1 / 

(2X,89HDATA»(F1*)G(F2*), (Fl*) AN3 (F2*) ARE \/ARImaX OF FI 
HCDE AND SECOND rtuDE R E SPt CT I Vt L Y/ /iO X, 1 i, 9 1 1 1 / / ) 



X2MFA2 
X2HFA2 
X2MFA2 
X2MFA2 
X2MFA2 
X2MFA2 
X2iFA2 
X2MFA2 
X2MFA2 
X2MFA2 
X2MFA2 
EXTEND 
X2MFA2 
X2«FA2 
X2WFA2 
X2MFA2 
X2«FA2 
X2H.>-A2 
X2MFA2 
X21FA2 
X2MFA2 
X2«FA2 
X2«1FA2 
X2MFA2 
X2MFA2 
X2«<FA2 
X21FA2 
X2MFA2 
X2.^FA2 
A2MFA2 
<2^FA2 



ERIC 
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c 
c 
c 
c 
c 



c 
c 
c 



50 
60 



Sia><OUTlNt VAKl>1X(NtiJ»<i,N4CQLS»NVFCri) 

CuinON // A(700,lO),f>(10),H( 700)#FlT(14),v(10),S(700,iO) 
L » T<10#10) , TTdClO) 

«fcAL RJuT( 10)»VAR(10),C!j.*«V(10) 
VARIMAX f^ACTGR kOTATIQN 

OUTPUT COMIUNALITIES AND NORMALIZE FACTORS 



sun - 

OQ 10 
10 SUM ■ 
hRITE 
WRITE 

Sum ■ 

OQ 20 
hRITfc 
SUN - 

20 H(I) - 
WRITE 
DC 30 
CO 30 

30 A(IW) 
KRCT ■ 

<»0 KX « 1 
KZCT - 



0.0 

I ■ 1»NCULS 
SJfl^Vd ) 

<6»260) NCCLS»SUM 

16»270) 

0.0 

I - i,NRavs 

(6»2d0) I»H(I) 
oUi*H( I) 

SaRT(H(I) ) 
(6,290) SLi 
J ■ ItNCULS 
I ■ 1#NRUWS 

« A(I,J)/H{I) 

(SCHS»(^<C0Li-l))/2 



COMPUTE Sums A»B»C ASD 0 



KY - 

CONA 
CONb 
CONG 
C-JND 



KX + 1 

■ 0. 

• 0. 

■ 0. 

• 0. 



l/A^lMjf 

EXTEND 

<2M-CC«*. 

.AkIM2 

1/ARIM2 

\/ARlM2 

VARIH2 

VARIM2 

VARIM2 

VAKIM2 

VARI»'2 

VAkIN2 

^ARIM2 

VARIM2 

VARIM2 

\/ARiH2 

k/ARIM2 

\/ARIM2 

VARIM2 

•/ARIK2 

VARIM2 

VARIM2 

VARI«<2 

VARI>12 

VARIM2 

tfARIM2 

VARIM2 

1/ARIM2 

VARIM2 

VARIH2 

wAKiM2 

VARn2 

V AR IM2 

vAKlH2 



ERIC 



00 70 I - l,NROkS 

U - <A(I,KX)+A(I,KY))*(ACI#KX)-A(I,KY)) 
V • 2.0*A( I#KX)*A(I,KY) 
CCNA - CQNA+U 
CQN3 - CQNd^V 
CQNC - CONC + (U + V) ♦( U-V) 
70 CONO « C0ND+2.0*U*V 
FLN - FLOAT(hROWS) 
TNUM - C0N0-(2.0»cnNA*CCNB)/FLN 

TOEN - CONC-( (C0NA+C3N3)*(Cu)NA-C0NtJ) )/FLN 
ANUf« - ABS(TnUM) 
ADEN - ABS(TOEN) 
If (ANUM-AOEN) 90,80,100 
80 IF UhUJI.EQ.O) GC Tu 180 
GO TO 105 

C 

C hUH LESS THA»J DEN 

C 

90 TiAN4T - AnUI/ADEN 

IF ITaN^T.LT. 0.06993) GO TO 190 
C0S4T - 1.0/S0RT(1.0*TAS4T»*2) 
SIN4T - TAN4T*CrS4T 
GC TO 110 

C 

C NUM GREATER THAN OEn 

C 

100 CTN4T - AOEN/ANLM 

IF (CTiN4T.LT. 0.06993) GC TO 200 
SIN^T ■ 1.0/SWRT{1.0*CTn4T»*2) 
COS^T » CrN47»SlN4T 
GQ TO 110 

C 

C NUM EQUAL OEn, fcLlH Nln-ZERC 

C 
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\/ARI«2 

VAfclM2 

\/ARIM2 

\/ARIH2 

\/ARIH2 

VA«IH2 

VAkIf«2 

VARIM2 

</ARIM2 

\/AKI^12 

VARIH2 

VARIM2 

VARIH2 

VA.<IM2 

VARIM2 

VARI.i2 

VARIM2 

VARIH2 

VAklM2 

>/ARIf<2 

VARIM2 

VARIM2 

1/ARIM2 

VARIM2 

VARIM2 

\/ARIM2 

VARIH2 

VARIM2 

I/ARIH2 

VAkIM2 

VARIM2 

V4R IM2 

\/ARI»«2 

VAK I«'2 



lOb SIN4T • 0.707107 
C0S4T « 

c 

C CCPUTt ANGLE CF AOTAIICN 

C 

110 CCS2T • SwRl(0.5*0.5«CUSn) 

SIN2T • S1H4|/(2.0«C3S21) 

COST • SQRT(0.S>0.$«C0S2T) 

5IKT - SlN2T/t2.0«CQST) 

IF (TOEN.LT.O) 6C TC 12C 

COSPHI • COST 

SINPHI • SIM 

GO TO 130 
120 CQSPHI • 0.707107*(CQiT*SlNTI 

SIMI»HI • ABS( 0.707107*(C3iT-SINT» ) 
130 If CTNU^.LT.O) ilNPHI • -SInPHI 

C 

C ROTATE FACTuRS 
C 

140 00 150 1 - l>^RatlS 

t ■ AII*KX)*CLSFhI+An»Ky)»SI.>iPHI 
A(I#KY) ■ -SINPHI*A(I#K)()*A( I,KY)*CCSPHI 
150 Afl^KX) ■ U 

160 IF (KY.EQ..'^CDLS) Gd TO 170 

HY • KY*1 

GO TO 60 
170 |<X • KX*1 

IF CKX.ihE.NCOLSI go TO 50 

IF (♦(ZCT.ME.KRtTI GJ TO 40 

GO TO 205 

C 

C NttH ANJ UEN tfcUAL ZERO 

C 
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rfARIM2 
/ARIM2 
tfAtiIK2 
VARIK2 
¥ARIK2 
rfAftIM2 
VARIHZ 
VARIM2 
VARIH2 
VAkIH2 
VARIM2 
VARIf«2 
\/ARI«2 
VARIH2 
VARIH2 
VARI«12 
VA«I?«2 
VARIMZ 
VARI*»2 
VARIf12 
WARI*«2 
VARIM2 
VARIX2 
VAkI*12 
VAKIH2 
VAKIM2 
WARH2 
VARIM2 
VARI-'Z 
rfARlMZ 
VArI««2 
VAki''2 
VAR IM2 



180 KZCT " KZCT*i 

GO TG 160 
190 IF (TDEN.GE.O) GJ T'J 130 

CrSPHI - 0.707107 

SiNPHI ■ CuSPHI 

GO TC 1^0 

C 

C DES / -ib** LE>S THAr. E 

C 

200 C0S4T - 0.0 
ilN4T • 1.0 

GO TO 110 

C 

C CUTP'JT FACTORS 

C 

203 DC 220 J - 1,^CC.L-] 

ROJT(J) » 0.0 

OC 21C I ■ l,Ni(<C»»5 

Ad, J) ■ A(I,J)»H(I) 
210 RC-:)T(J) - jCT{ J)*A( I, j)*»2 
220 VAR<J) - R.JjK J)/FLN 

*KITE (6,300) (1,1 - l,Nv;-,L:i) 

Ci 230 I « IfNROw:) 
230 WRITE {6,31J) I,(A(I,j),U • i,^■C•:.L^) 

WRITE (6, 320) (RL:T(J),J « If.iCjLM 

WRITE (6,330) («AK(j),j « 1,NC0LS) 

CllV(l) » ^AK(l) 

DC 2<»0 I - 2,NCQL£ 
2<fO COMVCl) - VAK ( I ) tCu-v V(I-I) 

l»RITE (6,340) (Cij.'VtJ),o • l,N|CULi) 

IF (NVMCH.LE.O) ktT-.V.s 

00 ZbC I » l,NRnw', 
2t)0 FLNCH 3 50, K(A( I,j),j.i,NCJLj) 

KiltRN 



VARM2 
VARI^Z 
</AR IN2 
VAkll? 
VARIf«2 
wARIr'2 
^ARI M2 
rfARlM2 
tf ARH2 
^ARIM2 
\/ARIM2 
yARI>«2 
VA^I*'2 

yARIf«2 
VARI-2 
VARIH2 
(/ARI-2 
VAkI«2 
VAR IM2 

V Ai< 112 
vAR IM2 
>/ARI«2 
»/A,< IM2 
»/ARI>'2 

VARI«'2 
VAR I»1? 
VAk I M2 
vAR I-3 
vARI'12 
« AR 1".^ 

VAR 1-2 

V AR I^.Z 
^ R I " 2 



c 
c 
c 



TH6 FUST»I3#9H fiOJIi 
C3MMINALITIE S# //) 



FORrtAl bTATEKEMS 

260 FQR-^AT (21H-THE SU»- JF 
270 FQRhAT (26H0 KQ. 
280 FQRrtAI (IH #I6»F1^.^) 
290 FOPMAT (2bH0THE $Ur* OF C GlfJNAL I TI t S IS,Fi^.M 
3C0 FCRMAU*! VA«*,1 3X, »y/ARlr. A> FACT3R L f A 0 1 NGi ♦ / 3X , •n 
310 FOR»^AT (H» 15, 3X,10F11 .3) 
320 FORMAT ( 1H-, 8HVAR lANCE » lOF 1 1 . 3/ ) 
330 FORi^AT (lX#8HPCr yAr< #10F11.3/) 
340 FORMAT (lX»8HCU.»* PCT »10F11,3) 
350 FiiRrtAK I7»9F8-t>.M 10F8.i>)) 
END 



li#F14.<./// ) 



♦fid, 9111) 



S'JP^O'ITI jc ^PORTC^-Si,) 

CALL oYSTtM(c;;>,i AIJC'''r.) 
STOP 

E*-T}Y AyU' T'' 

CALL ^vsrrH(5y,f|^r.) 

SL'SRfL'TlNb Ii>K.N,T ( NA^-t ) 
Pf<r(7 1,MA*<E 

Ca'll'em? ■ '''''''' ^^-t MIKICESM 



^AkI*-? 
VAR IM2 
VARi*<2 

VARIM2 
VARiM2 
rfARIf'2 
VAR JM2 

V *.RI»12 
VARI*«2 
t/AR IM2 

VARI-2 
</ARI««2 



A- JRT 

A' JRT 
A- JRT 
A' JR^ 
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SUiiKGoTl^JE ''5Aa{A,KRA,NCA, J,NRo,NCE»C»>J^RA, >iN<Ra#NNRC) 
REAL A(NNRA,U, B(».^R3»1), C(r.NRC,l) 
C fISAS - C « A*6 

C S.I. JAsIk - 3/10/71 - OL 2 kCwS AT a IMF 

7.F( nCA .NE. NP6 ) CALL <»hriAB ) 



IF( NRA .t3. 1 ) Gj IQ 2tj 
^2 • 2*(NRA/2) 

00 20 J • 1,NCB 
OD 20 I = l,Ni:,2 

SI • 0. $ S2 » C. 

00 10 K « l^'^CA 

SI • SI ♦ A(I,K)*b(K .0) 

10 S2 « i2 ♦ A( 1*1, K )*b(K, J) 
C(I»j) » 31 

20 C(I*1,J) » j2 

If< NftA .£t. n2 ) RLl!J«\ 

25 00 ^0 J « l,NCd 
51 • u. 

DC 30 K « 1,NCA 
30 SI » U ♦ A(NRA,K)4b(K, J) 
^0 C(.MRA,J) = SI 

RETHk.N 

E'vO 



ERIC 
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f^EAL A(HN»A*NCA) , B<NN«B»».Cei » C(NNRC#MCBI 

♦ "^SATB - C • TRA^SP0SE{A)*8 

IF<NRA.£(j.NRB) GO TL 2 

CALL IP»NT( SHMSATB I 

2 OL '♦O I • 1#NCA 

00 <»0 4-1,nC8 
S « 0 

00 30 K • 1,NkA 
30 S » S+A«K, Jl 

40 C<I,J) • S 

RtTJRM 

EnO 

SlSaruTlNE '•SABT U»^RA,NCA»B»^P3»NCa»C»>^^RA,^NR^.^K^RC) 

REAL A<NNRA,^4CA) » 8 (NMR|}»KCb> , C ( ^sRC t SCa ) 

IF( NCA .E9. see ) GLi FQ 6 
CALL IPRNK 5HMiAdT ) 

6 DC- 60 I » 1»NRA 

0" 60 J«l»>:Pd 

^ ^ 0 

JO 50 K • 1,^CA 
t>0 S « S*A( I,K >*B( J»K) 

60 C(I»j) « b 



END 



ERIC 



APPENDIX F 
THE HfMS "MATRIX" PROGRA>I 

The HSMS MATRIX program was designed by Edward Friedman to 
nake it possible to create printed displays of selected task data. 
These are used to aid in the assignment of tasks to job levels and to 
design curricula. 

DESCRIPTION 

matrix's main function is to array the scale values for 
skills and knowledges as rows, in the selected order in which they 
appear in tasks (which are the columns) . This permits inspection of 
"profiles" of task requirements and comparison of task requirements. 

In the array the tasks are the columns, listed by Task Code 
Number. The skill and knowledge categories, listed by their 8-charac- 
ter or S-digit code identifications are the rows. The tasks are listed 
from right to left in the order designated hy *C UPDATE cards. The 
skill and knowledge categories are listed in the order in which they 
appear in the tasks in the array. 

The entries in the matrix are the non-zero scale values of 
the variables re.-^uired by the tasks. Zero values are not shown, and 
the decimal points are omitted. Possible HSMS entries are 10, 15, 20, 
25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, and 90. The arrays 
can be ordered in expanded or compressed form and in multiple copies 
for varioi'S uses. 
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The MATRIX program also provides data "dictionaries." One 
lists the tasks in their order in the array by internal number and by 
Task Code Number along with an abbreviated task narre. Another lists 
the skill and knowledge categories vith the skills first, followed by 
the knowledge category 8-digit codes in ascending numerical order. It 
also shows the frequency across the tasks in the display. 

MATRIX is written in FORTRAN IV and was used in the Control 
Data Corporation's (CDC) 6600 computer at the Courant Institute of 
Mathematical Sciences of New York University, The operating systems 
in use during HSMS analyses were KRONOS and NOS (Network Operating Sys- 
tem), MATRIX was stored on magnetic tape in compiled and loaded form 
(i.e., in binary object code in non-relocatable form) and in OLDPL 
form. The HSMS task data are transferred to magnetic tape in the form 
of an OLDPL. HSMS maintained the decks in numerical order ia the OLDPL. 

A utility program, UPDATE, is used to generate temporary lo- 
cal files in the form of a *C tile which is part of the input file for 
MATRIX. The input file for UPDATE is magnetic tape in OLDPL form, and 
the output file is K=TAPE2. The input file for MATRIX is TAPE2, and 
the output file is printed. 

SYSTEM SUBROUTINES CALLED BY MATRIX 

The MATRIX program presented in this appendix calls a series 
of subroutines. Among these, EXIT, TIME, and DATE have not been in- 
cluded in the listing, since they are systems-based. It is assumed 
that the user can utilize comparable routines after reading the de- 
scriptions which follow. 

F-2 
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EXIT 



This subroutine terminates program execution and returns 
control to the operating system. A STOP statement may be preferable. 

TIME(a) 

This subroutine can be used as a function ^r subroutine. 
The value is returned via the argument c^nd the normal funccijri rt^turn. 
The subroutine returns the current reading of the system deck as the 
value of the argument a or of the function in the form lOHhhh .mm.ss.b, 
where b denotes a blank, and hh, itj*, and ss are ?he numbers of hours, 
minutes, and seconds, respectively. The value returned is Hollerith 
data and can be output using an A format specification. The ?:yDe of 
this function is real. 

DATE (a) 

This subroutine can be used as a function subroutine. See 
CALL TIME(a), above. The current date is returned as the vaJ ue of ar- 
gument a or of the function in the form lOHbmm/dd/yyb (unless it is 
changed at installation option), where b denotes a blank, mm is tht 
number of the month, dd is the numbfjr of the day within the month, and 
yy is the year. The value returned is Hollerith data and can be out- 
put using an A format specification. The type of this function is real. 
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STRUCTURING THE INPUT FILE FOR MATRIX 

In the set-up presented below, the NOS stem in use in 1977 
at the Courant Institute's CDC 6600 is assumed. It is also assumed 
that the reader can refer to the UPDATE Reference Manual or a counter- 
part program. 



INPUT FILE FOR MATRIX 



Name 


Keypunch Cards (one per box) 


Instructions 


Svs- 
tems 
Cards 


MATRIX 


Identification number and 
other information depending 
on system. 




UD£«K^ ) 


USER and user code. 




CHARGE ( \ 


unarge card . 




RESOURC fMT=2') 


uii ly i i mu Lc tnan one sec oi 
Labis, uaca lui ru\ii\iA array 
is being run. 




HEADER. MATRIX Pfr 


upLxuiidi lu pnnL ouc neacmg 




LABEL (OLDPL , VSN=T ) 


xuciiLxixcb uliUi^Ij cape. 




UPDATE(Q,D,8,K-TAPE2) 


For selecting tasks using *C 
form of UPDATE. 




UNLOAD C OLDPL') 


uiiiy 11 jusc one sec or task 
data is beinE run 




LABEL (PROGRMS,VSN=T ) 


Identifies program tape. 




SKIPF(PROGRMS,A) 


Skips over EDIT, PCVARIM, 
X2M0FA, and X2MFA2. 




COPYBF(PROGRMS .MATRIX) 


Copies program to local file. 




UNLOAD (PROGRMS) 


Only if just one set of task 
data is being run. 




RFL( 200000) 


Defines field length. 




SETTL(IOOO) 


Time limit. 




MATRIX. 






End of Record Card 


Only if just one set of task 
data is being run. 




REWIND ( OLDPL , TAPE2 , MATRIX) 


With second set of task data* 




UPDATE(Q,D,8,K=TAPE2) 


With second set of task data. 




MATRIX. 


With second set of task data. 






*These three cards are repeated 
again in this order as many 
times as there are sets of 
data beyond two. 




End of Record Card 


After last systems card. 




INPUT FILE FOR MATRIX (continued) 



Name 


Keypunch Cards (one per box) 


Instructions 


In- 
put 
Cards 
For 
UP 
DATE 


^ > > » 

and/or 

*c, 

*c, 

etc* 


Calls the first set of tasks to 
be placed in the first array in 
the order selected; i.e., HSMS 


uses point ccore order. Task 
Code Number order, or factor 
loading order. 
Col. 1 is alv;ays asterisk; 
Col . 2 is always C ; 
L.01 . J IS always comma, 
then follows one or more Task 
Code Numbers separated by com- 
mas, with no comma after the 
last entry, up to Col. 80. It 
is convenient to place tasks 
uii carcb so cnaL cney can oe 
rearranged for various arrays. 


End of Record Card 




In- 
put 
Cards 
for 
MA 
TRIX 

Para- 
meter 
Card 


MATRIX etc. 

etc . 


Title card. Cols. 1-40 read 
as one line; Cols. 41-30 read 
as second line. Use to iden 
tify the salient features of 
this first array of task data. 


Cols. 3-5 
Col. 10 

Col. 15 
Cols. 16-20 


Right justify in fields except 
as otherwise indicated. 

Number of tasks in this set; 
up to 500. 

Punch 1 for long, expanded 
array. 

Punch 2 for short array. 
Punch 3 for condensed array 
(best tor analysis). 
Punch 4 for all of the above. 

Number of copies of output. 

Leave blank to call input data 
from UPDATE TAPE2 or enter name 
of alternate file. Left justify. 




End of Record Card 




Repeat the set-up on this page for second set of tasks, ^.ad repeat again 
for any additional sets of task data, one set-up per set. 


1 End of file card comes last of all. 



F-5 

251 



N/\ATR\y 

t!VERLAY(HATRlX,0,0) 
*• TAPE 7»0UTPL'T ) 

IMEG6R TASK, STABLE »A»TID»TA3LE» CHOICE, FR£U»PTk 



COIhON /GILO/ 
COMMON /GILI/ 
CCMHON /GIL2/ 
CC'«<MDN /GIL3/ 
♦ FRECK bOO),FT.?{500) 
COMMON rtAT(40000) 



TI ILE (8),TCOAr»NPG,7if* 
TAbK( !>00),STA3Lt(l6),A(8) 
NTASK,IiRT,NS V» N» NR, NK C» T I D, CHOIC E, NC OP 
KC(500),KV(500),TAeLE(500),LCl 500),LCS(501),NB{500) 



CALL DATE (TOOAr) 
CALL TIflEtTI'l) 
5 CALL INPUT 
CALL HEAO 

ISRT«0, TASK DATA S'JRTE3 AS OESIREO AND li uN MIT 2 
ISRTi»0, DATA rUST BE EM-^ACTED FRu" MASTER FILE ON l'>»ir 1 
AND >,ILL dE »RIITEN JN UNIT 2. 

IF( ISKT.nE.C) Call SELECT 
CALL GET 

CALL FORM 
CALL OU' 

GC TO t> 

eno 



77/09/15 



KROnCS 
MAfRU 
1ATRIX 
iATRIX 
C3.iMCN 

'cXIEND 

C':'«i*'C»» 

EHIEND 
EXTEND 
EXIEN3 
MATRIX 
iATRIX 
MATRIX 
-iATRIX 
MATRIX 
1ATRIX 
MATRIX 
•-ATRIX 
lATRIX 
IATkIX 
MATRIX 
MATR IX 

ilATRlX 
MATRIX 
MATRIX 
MA TRI X 
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ERIC 



SUBROUTINE INPUT I.>i;>UT 

INTEGER TASK»STABLE#A#TIO,TAdLE,CHClCE#FRtw#f'l(< C JWhON 

CCHMON /GILO/ Til LE < 8 ) #T OOAY, NPG, TIM C)«i5>« 

C01«0N /GUI/ TASK! 500), STABLE <16)#A(8) 6x(£-40 

COMMON /GIL2/ NTaSK* I SRT* N5V, iM# N»» NKC# TIO#CHaiC £. NCCP CuK»»^^ 
COHMON /6IL3/ KC{500),KV<500),TA6LE<500),LC<500),LCS<t»01),NB(bOO), tXfEND 

♦ FREQ(500),PTR(500) E<TtND 

COMHON MAT <40000) E<TENO 

C INITIALIZE HAT WITH BLANKS ImPOT 

call SETRAf (hAT#'»0000# 1H ) cXTEMO 

NPG-0 iNrJlJ 

C IMPlT 

C READ IN TITLE CARD lNt>UT 

C Hf'oT 

5 REA0{5#5OO)TITLE ISKUT 

I^CEOFC 5) )10#20 ISPLT 

10 CALL EXIT iNf-LT 

C INPUT 

C READ IN PARAMETER CARD In^I-T 

20 REA0<5#510)NTASR»CH0lCE,NCnp,Ii.Kr InP'jT 

N-NTASk IMPtT 

IFdSRT.E&.O) GP TO 3C InPUF 

C READ In task NGS. I-Jf-LT 

C IF DATA li Tff BE EXTt<ACTEO fRZh "«aSTEk FRt J^ U>»II 1 I^PtT 

REA0(5»520)<TA5i<(I),I«I,n) Ixt-uT 

30 RETURN iMi^-UT 

C INPUT FJR.IAT STAfcMLMlS iNr^T 

500 FORf<AT{ 6A10) IN,^L'T 

510 F0Rf'AT(I6I5) I>.PbT 

520 F0RP.AT(5X15I5) :;i-.T 

END I'.KuT 



ERIC 
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c 
c 
c 



SUBROC'TINE bELECT 

INTEGER TASK»iTAdLE,A,TlD»rAbLE,CHLICt,F^cw/-PlR 

COMMON /GILO/ TI I LE ( 8) , TJUAt* NPG» Tir 

COMMON /GUI/ TASK(50O),3rABLE(I6),A(8) 

COMMON /GIL2/ NT A SK , I i RT » NJS V» N» NK , NKC > fl 0» CHo IC E » SC C f 

COMMON /GIL 3/ KC ( t>00 )» KV ( 300 ) » T A8 L E ( 500) » LC ( bOO ) » LC3 ( 501 ) , nB ( 500 ) , 
♦ FREC(500),?TR(500) 

COMMON MAI ( <<0000 ) 
SELECT ONLY TH3SE TASK CAROi DESIRED 
IN ORDER SPECIFIED ON IN->UT TASK(S) CARDS 

DO 100 I»I,N 

RE»ilNO 1 
25 ReAD( 1,1000) TI0,NCiJ» A 

IFiEOhd) )30»40 
DATA EXHAUSTED AND TASK NCT Fn-JNO 
30 CALL ERROR(l) 
40 IF(TASK< I) .EQ.II D) uj TC 60 
SKIP ADDITIONAL CARDS 

CO 50 J«1»NCD 

READ( 1, 1010) 
50 CONTINUE 

GO TO 25 
WRITE SELECTED JATA J4Tu' UNIT 2 
60 WRIT£(2»1020)A 

DO 70 J«1»NCD 

READ (1,1020) A 

*RITE(2»1020) A 
70 CONTINUE 
100 CONTlNUt 

fMT 2 CONTAINS DATA In ..t^CtK REtuUED 



SELECT 
C JlnGN 
C?Mf^CN 
EK rEf^D 
C JM«r)>4 

cKTEND 
E<TENi) 
EXTEND 
icLECT 
ELECT 
,cLECT 
SELECT 
SELECT 
SELECT 
SELECT 
SELECT 
SELECT 
SELECT 
jELbCT 
SELECT 
SELECT 
iELECT 
jELECT 
SELECT 
SELECT 
oELtCT 
iELECT 
iELECT 
.^^lECT 
icLECT 
iELECT 
>£LECT 




ENOFILE 2 

RErfINO 2 SELECT 

RETURN SELECT 

C FORMAT CARDS FOR TASK DATA SELECT 

1000 FORHAT(r5,l6,T29,I2,Tl,8A10) SELECT 

1010 FOf.HAT(lX) "LECT 

1020 FORMAT(8A10» iicLECT 

END SELECT 

SE» ECT 



GET 
110 



1 



115 



SUBROUTINE GET 

INTEGER TASK, STABLE, A, T ID, TABL E,CHOI CE, FREiJ, PTR 
COMMON /GILO/ TITLE(3),TQDAr,NPG, TIM 
COHHON /GILl/ TASK(500), STABLE(16),A(8) 
COMMON /GIL2/ NT ASK, ISRT, NSV, N,Nk , NKC, TI D* CHQ iC E, MCOP 
COMMON /GIL3/ KC ( 500 ), KV S500 ) , TAB L E( 500) , LC ( 500 ), L CS ( 50 1 ) , N B ( 500 ) , 
♦ FREQC 500) , PTR ( 500) 
COMMON MaT(^OOOO) 
INTEGER 3CD(5) 
REMIND 2 

DATA FROM UNIT 2 AND FUR*f MATRIX MAT 
00 200 I»I,N 

READ(2,2000)TID, NCD,BCD, TASKJ I) 
IFIE0F(2) .EQ.O)GO TU 115 
WRITEi7,l) 

♦J°mI!lEJErUR5^r' ^ ^'^'^ ^'^^ "-^^^^ ^'^ 

CALL EXIT 
CONTINUE 

IFC (MOD( I, 25)-I) .NF.O) Gd TC 120 
CALL TOP 
WRITE( 6,6000) 



GET 

t <TEND 

EXTEND 

tXT E^0 

EXTEND 

GET 

GET 

GEl 

GET 

GET 

EMEND 
Ex TEND 
EXFEND 

Extend 

FxTtND 

hi) 
GET 
GE i 



ERIC 
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120 i«RITE(t)»6100)I#TI0*3C0 GET 

H«l GET 

L«MSV GET 

REA0(2«2010)IT»fKV(J)»J>K»L( GET 

IF(IT*NE.3RT01) STOP GET 

IF(NCO,E0.1) GU TO 180 GET 

C (NLY CARD TJC AND TOl EXIST GET 

C LBTAIN KC INFO GET 

130 DO 150 II»2*NCu GET 

|<«L*1 GET 

L»L*^ GET 

REA0(2#2020)(KC( J)#KV(J)# J»K,L ) GET 

150 CONTINUE GET 

180 NKC-L GET 

190 CALL UPOAfEtn GET 

200 CONTINUE GEI 

CALL Tup get 

hRITE (6,6200) N,NR GET 

RETURN GET 

2000 FQRMAT(Tb#R5#T29# I2#T32#'»A10/A7#T5#I3) GET 

2010 F0R*<A1(T1,K3,T31,16(IX,R2)) GET 

2020 FaR.HAT(T23,<»(2XKe,lXR2) ) GET 

6000 FORMAl { /10<«I»,7X*TASK NO.*,bX*TASK IDEnTI F ICAinN*, GET 

X /lX#8(10ri )/) GET 

6100 F'JR^^AT(/ 6<I5# 9xk6, 3<'»A10# A7) GET 

6200 FORMAT ( IHO* 5x«t^ .'. TAiK5> « ♦,I6,5a»Ni:. jr OAItGLRIEb * ♦,I6) GET 

£N0 GET 



ERIC 



c 
c 
c 



SUBHQLT1N6 UPOATEIJ) 

INTEGER TASK, STABLE* A, TID# TABLE* CHOICE, FREC#PT ft 
COMMON /GILO/ TITLE ( 8) , TODAY, .SfG, TI** 
COMMON /GILl/ TASK(500»,STABLE(16),A(8) 
COMMON /GIL2/ KTASK, ISRT,NSV, S,NR, KKC, TID, CHOICE, NCL'P 
COMMON /GIL 3/ KC ( 500 ), KV ( 500 ), T ABLE ( 500 ) ,LC ( 500 ), LCS ( 501 ) ,Nb ( 500 ) , 
♦ fRE0(500),PTR( 500) 
COMMON MAT(<>0000) 
INTEGER SABLE(500),SAT(500) 
INTEGER 0Z,B8 
DATA DZ/2R00/ 
DATA Bd/8R / 
IFIJ.NE.I) GO TC 150 

INITIALIZATION ENTRY FQR FIRS! TASK GNLt 

CALL ZERO(FREO,NTASK) 

ISV-0 

IKC-0 

00 100 I»1,NKC 

IF( I .LE .NSW) GO TO 90 

IF(KC(I ).Ea.B8) GG TO ICO 

IFIKVU) .Eli.DZ) KV{I)»2R 

IKC-IKC+I 

SA8LE( IKC)-KC(I) 

KAT(IKC)»KW(I) 

GO TO iOO 
SKILL CATEGORY AND VALuE 
90 IF(KV( Il.eQ.DZ> GO TO 100 

ISV-ISV*) 

TABLE(IiV)»STA8Lb( I) 

FREC(ISV)-1 

MAT( ISV)»K\/(I) 



J^DATc 

C 3'•^«C^ 

CDMKHN 

EXTEND 

C3,n*'DW 

EkTEND 

EXTEND 

EKTEND 

E/iTEND 

UPDATE 

vjODATE 

u'>DATb 

vJ^OATE 

o^OATE 

UPDATE 

'jPOATt 

J'DATE 

UPDATE 

JPOATE 

UPDATE 

UPDATE 

O^^DATE 

UPDATE 

UPDATE 

UPDATE 

U''DATE 

'JPDATE 

UPDATE 

UPDATE 

UPDATE 

UPDATE 

L->OATE 

u-DATC 
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c 
c 
c 



100 CONTINUE 
K-ISV+IKC 
LC(J)-K 
NR-K 
KT-K 

SORT IN ASCENDING ORDER <NO<i(.tDGE 
IFdRC.EO.O ) GC TO 120 
PTR(1>-1 

IF(IKC.EQ.l) GU TO 105 
CALL SriLS«T(SABLk,lKC»PTR) 

105 DO 110 I«1#IKC 

TABLE{I*ISV)-SAELb( PTft( I) ) 
NAT( I*ISV)-NAT(?TR( I) ) 
Ff(EU<i*ISv»-l 

110 CONTINUE 

12 0 RETURN 

Entry fcr all 



CATEGORIES FOR FIRST TASK 



TASKS EXCcPI THb FIRST 



150 KQ-NR 

K-KT+KR 

00 200 I-1#NKC 

IF((I.Lt.NS^».ANO.(KV(I).£U.DZ)) 
IFd.LE. NSV) KC(I)-S ABLEC I) 
1F(KC( I) .EO.be ) 3 J TO 202 

IF(Kv(I).tQ.DZ) KV(n-2R 



GC- lu 200 



160 



00 160 L"l*t<G 
IF(KC(I).£w.lABLfc(L)) Gc 

COnTINLl 



Tf. 17C 



UPDATE 
UPDATE 
UPDATE 
UPDATE 
UPDATE 
UPDATE 
UPDATE 
UPDATE 
UPDATE 
UPDATE 
UPDATE 
U'>DATE 
UPDATF 
UPDATE 
UPDATE 
UPDATE 
UPDATE 
UPDATE 
UPDATE 
UPDATE 
UPDATE 
UOuATE 
UPDATE 
JPOATQ 

UPDATE 
UPDATE 
UPDATE 
UPDATE 
UPDATE 
UPDATE 
UPOATf. 
UPDATt 
UPDATE 



2 



bo 



ERIC 



c 
c 
c 



NO ENTRY EAlSTtO IN TAdLE Sf-'rCRE aOD &N£ 

SA8ie(« )»Kc<n 

GO TO 200 

EHJRI EXISTED IN TABLE d6FC^6 UPDATE APPKDPiaTd kuw FoR THIS TASK 

170 CONTINUE 

KTPL » KT ♦ L 

IF CKTPL.Ge.--.OCOO) GG TO 220 
f*AT<<TPL5 - KVd ) 
FPEQ(l}»FREC{L)-H 
200 CONTINUE 

C 

202 IF{H.EQ.O) GD TO 210 
PTRC1)-1 

IF(fl.EQ.l) GD TO 20^ 

C 

C SORT IN ASCENDING UROER N<:W CATE63RIES FOf? IHIS TAVk 
CALL SHLSRT(SABL£,M,PTR) 

C 

20^ 00 205 I»l,« 
NR-NR+1 

TA8LE(NR )- SABLE{PTR{I) } 

FREOINR) »1 

K"K+1 

rtAT(K)-NAT(PlR(I) ) 
205 CONTINUE 
210 LC(J)»NR 

KT»K 

RETURN 
220 WRITE{7,230» TID,J 

CALL AdCki ( lOH-iVEKFLu* ) 
230 F0RHAn5K* LAST fA.n. kEAJ »,Ke, • CF ♦,IiO,*|ASKS kU.)*/J 
E N J 

ERIC 



UOQATE 

UPOATF 

UPDATE 

UPDATE 

U«»OATt 

UPDATE 

UPDATE 

UPDATE 

EXTEND 

EXTEND 

EXTEND 

EXTEND 

uPD/TF 

JPOATt 

JOOATE 

U-JDATE 

UPDATE 

UPDATE 

UPDATE 

UPDATE 

UPDATE 

UPDATE 

UPDATE 

J-'OATE 

UPDATE 

UPDATE 

UPDATE 

UPDATE 

UPDATE 

UPDATE 

UPDATE 

J-^'JATE 

L < fcND 

t» TEND 

r <TEnD 



fir' f^ 



Sl-BRuuTiNc Fjk>* f --RM 

lNT£Gti< TAS^,StA6Lr;,A, rn»TA3Lt»CH;.ICr»f-«cw, k|k C jrtrtON' 

CJrtftON /GILO/ TlUtJ 3),TuDAT»NPG#Tl«' C3'<MnN' 

COMfCN /GUI/ TAbK{50U),:>TAttLE{l6)#A(^) £XT£nO 

COnrtOs /&IL2/ ^^AS^»IS«<T» 4iV»\>r'R#NKC.TI0*CHGiCE»NC0F Cli^O* 
CJMIOn /GIL3/ KC(500),KV(500),TABLE(^jOO)»LC(500), 105(301), N8{500), ExTt^D 

+ FREU{ 500) »PTK( 500) bXTEwO 

MAK^iOOOO) EXTEfJO 

KALL -srASK*! fjf(t. 

OQ 100 I»2».\ALL p.j^i,' 

LCSU)«LCS{ I-i) + LC( I-l) FQk*" 

iOO CONTINUE frnf» 

150 RETURN p irf^ 



InTcGEk TA5^>5rABL£, a, ri:,TAaLE,C.luICc,'^K£., -Ik 

Zl-'-t*jt. /5ILC/ TIlLt(8),T30AY,-lPG, I I': - •-.r'.^. 

C:>'ijN /GUI/ TA.K {5J0),iTAtL£< l6),A(b) T^TiLU 

CO^MjN /GIL2/ ^TA:(<.,I.R^.NSV,.^,^S,^.^C,TIO,C-. ICE,'sCrf ^ '^'s 

/'^L';v^o^^^'^p^;co^^^ --^^-^^ 

o: 10 i = i,NtiAi,K 

tNCCOe ( 3»oJl'j, T Ai< { I ) J TAjkU) , 

OtC':0£ ( 3; c020,^AS^( I ) ) U.^(I) . ', 



2uJ 



ERIC 



10 COHTIIlUE 

If ICCHOICE.LE.OI.e. CCHGICE.GT. 41)60 TO 300 

60 T0(10O,200»30C»40OJ CHDICE 
100 CALL HATOUT 

&0 TO 900 
200 CALL MATSHT 

50 TO 900 
300 CALL «ATCKO 

60 TO 900 
*00 CALL HATQUT 

CALL HATSHT 

CALL HATCND 

C PRINT FREQU6NCI6S FGR SKILL SCALES k^O KHQ*LtOGE CA7£GU-<IES 

C SORTED IN ASCENDING ORDER . 

900 CALL KCFftea 

CALL TI«E(TIH) 

CALL HEAD 

ENDFiLE 6 

DO 1000 ICQP«l#NCaP 
RE<rflN0 6 
950 Rf.ADi6»6000) LINE 

IF(EQF(6) .NEtO) GO TO ICOO 
kiRITE<7,6000RINfc 
60 TO 950 
1000 CONTINUE 
REWIND 6 
RETURN 

C 

6000 FQiiM4T (13A10,&6) 
6010 FORf'ATtn) 
6020 F0RMiT{R3) 



CJI 

JUT 

■IJT 

JUT 

OUT 

OJT 

Giil 

O'JT 

doT 

OUT 

OJf 

lUT 

JUT 

^JT 

•JUT 

JUT 

lUT 

JjT 

jJT 

JUT 

our 

OUT 

jur 

JJT 
uuT 
J'JT 
•3UT 
JJT 
OUT 
r-uT 
JJf 
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SU8.<0UTIN6 rATUiJT 

INTEGER Task, STABLE , A , TI D, TAB LE» CHilI C E» FR c C, P |R 
COH^ON /GILO/ TITLECaj^TL^OAr^SPGfTl* 
CDHKuN /GILI/ TAiK(50C),5TAt}LE{16),A(8) 
COrtMUN /GIL2/ NTASK»IiRT,NSv,N»NR,NKC»TID,CHUlCt,'*COP 
CQ»«>'ON /GIL 3/ KC (500 ), KV ( 500), TA8L fc ( 500 ), LC ( 300), LC 5(501 ),NB( 500), 
♦ FREt(500},PTR( 500) 
CQMM3N MAT (40000 ) 
INTEGER BLANK(25),LIN6 (25) 
h-25 

CALL SErKAY(BLANK,h,3R ) 
00 1000 K=1,NTASK»23 
M-»<lNO(rt»NTA!>l« ) 
nMK-24-(fi-K) 
00 9 0G L"1,Nk 

IF( (MG0(L»25)-1) .nE .0) G-2 TC 100 
CALL lOP 

IF(h«K.Eu.O) GG 10 60 

► RITE( 6, 6000 ) { r AjK { I ) , I«\, ,M) , ( 8L ANK ) ,J»l,*.hM 
G3 TO 70 
60 .rRITE( 6,6000) < TASK ( I), ) 
70 *»RITE( 6,6200) 
100 CALL bcrRAr( LIM,25,2k ) 
DO 150 I-K,"" 
II»I-(< + l 
120 J«LCS(I)*L-1 

130 IF (J.Gt .LC S { I) . Afi). J .L T .LCS U + 1 ) ) L T^iL ( II ) « i A i ( j ) 
150 CQNTlNOt 

2G0 wRITE l6,olOO)TAtLc (L) , L I nR , TaBL £ ( L) 
300 n<ITe(6,620J) 
900 CCNllNUc 
f.»l*25 



rtATClMT 

L 1M^0^ 
EXTEND 
C D^l. 
EaTENO 

Extend 

E<TEND 
.lATUUl 
1 ATLUT 
CATGUT 
A TOUT 
•-^Af'JuT 
"^AI LJT 
MAI I JUT 
1ATCUT 
•^AT :iUT 

^AluUT 
lAFCUT 
lATOlJT 
'l&TfUT 
'^AT jUT 
V.AT3LT 
>1A TuUT 
lATCUT 
MATQUT 
lATCUr 
."AT -LI 
•lAlCLT 

''at:ut 

lAT^'T 
-AT-.l'T 



erJc 



1000 CONTINUE 
RETURN 
C FORMAT STATEMENTS 

6000 F0RMAT(1X*CATEGQRY/»25(R3*/»),2X*CATE6jKY*) 
6100 F0RMAT(1XR8,* ♦,25(i5(f<2» *),ZKR8) 

6200 F0R«AT(1H ,i3(10h )) 

END 



MATCUT 
1 A TOUT 
"1 A TOUT 
rtATOUT 
MATOUT 



SUBROUTINE MATSHl 

INTEGER rASK» STABLE, A, TIO, T ABLE, CHOICE,FR EC, PIR 

COHMON /GILO/ TI 7L E ( 8 ) , TODAY, nPG, TI ^ 

COMMON /GUI/ TASK<bOO), jTABLE(i6),A{fa} 

COMMON /GIL2/ NT AiK, I SRT, S6 V, N, KR, NKC, TI 0, CHOI C E, NCOP 

COMMON MAT(40000) 

INTEGER 5LANK<25),LI>i£ (25) 

M«25 

CALL SErRAr(BLANK,M,3< ) 
DO 1000 K«1,NTaSK,2;> 
M»MIN0{ 1,NTASk) 
MI1K'24-(M-K) 
t/O 900 L»1,NR 
IF< (MGO( L,50)-l) .NE.O) 
CALL TOP 

IF(MMK.EQ.O) GO TC 60 

wRITE(6,6000) (lASKU),! 

GO TO 70 
60 WRI TE( 6, 6000) ( TAi>K { I ), 
70 i«RITE(6,6200) 



GO TO 100 



K, f".)* ( 6L ANK ( J ) ,J«l,»'fK) 



'lAIiHT 

CCf^NCS 

C ]«MOS 

E<TENO 

C jrfDS 

EXTEND 

EXTEN'D 

EXTEND 

MATSHT 

•^l ISHT 

^/.TiHT 

*1U5HT 

"<AT5HT 

^ATiHT 

fAT?HT 

'lAliHT 

nATSHT 

MA rSHT 

MATSHT 

•^ATI^HT 

f^-ATSHT 

1A1 SHT 



ERIC 
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100 CAIL SETRAy(LINE»25»2R ) 

00 ISO IsK#M 

II-I-K+1 
120 J«LCS(I)«L>1 

130 I^CJ.6E.LCS(I).AN0.J.LT.LCS(I*1)) LINE ( lU -(lAK j) 
150 CUNTINUE 

200 hRITE(6,6100)TABLe(L)» L INE »TAbL E( L ) 
800 CONTINUE 
900 CONTINUE 

1000 CONTINUE 
RETURN 

C 

C FORMAT STATEMENTS 
C 

6000 F0R«AT(lX*CATEGaRY/*25(R3*/*),2A*CATEGURY*) 
6100 F0RhAT(lXR8** ♦»25(1XR2* ♦)»2XR8) 

6200 FQRHATdH »13(10h )) 

END 



MAI jHT 
rtATSKT 
MAT5HT 
MATSHT 
iATiHT 
iAT6HT 
lATSHT 
^ATSHT 
1ATSHT 
lATSHT 
«ATSHT 
lATSHT 
MATSHT 
iATSHT 
iAISHT 
lAViHT 
.■•1ATSHT 
^'ATSHT 
^.ATSHT 
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50 



70 
100 



SUBROUTINE HATCnD 

INTEGER TASK,S'iABLE# A,TI0,1 ABLE, CHCI CE , FRE f>TR 
COHHON /GILO/ TITLEC 8) ,TJOAY,i>tPG,TII- 
COHf»ON /GILI/ TASK<500),irABLt(16)#K(d) 
COHHON /GIL2/ NTASK , URT, NS V# N, nR, NKC# 1 1 0, CHdl CE , 'iC OP 
COHMON /GIL 3/ KC ( 500) , ( 500 ), TABLE ( 500 ), LC ( 500 )# LCS ( 501 ), NB ( 500 ) , 
♦ FREQ(500), PTR(500) 
COHhON MAT(^OOOO) 

INTEGER 8LAMK(^2)#LINE (^2», vOoL( 3,200) 

CALL ZER0(KQ3L#600) 

H«42 

hH-42 

CALL SETRAY(BL&NK,f1M,3R ) 
DO 50 K-1#'<TASK 
DO 50 L-l#3 
LL-42+L*b 

K0OL<L,K)-( ( .N.hASKC 5^)) . A . SHI FT ( F AS k ( K ) , L L < ) . C . K 3 JL ( L , ^ ) 
CONTINUE 

DO 1000 K-l,NTASK,Mf. 
h-.1lN0(M,NTASK ) 
HhK«tf«)1-l)-(.1-K) 
DO 900 L»1»NP 
IFIL.NEU; on TO 100 
CALL POP 

IF(HHk.ECi.O ) GU I ) 60 

wi<ITEt6, 6000) ( ( (KCJLC LL, I ), 1-K, i) , IbLA.MM J) , J- 1, >''K ) ) , L L -1 , 3 ) 
GO TO 70 

wRITE(b#6000) ( (K.j.iLl LL» I), ) ,LL-1,3) 

».RnE(6, 6200) 

CALL StTRAY(LlNE,*'^,2K ) 

on 150 i»^,M 

II-I-K+l 



'iA rcNC 

EXTEND 
C 01*1 OS 
EXTEND 
EXTEND 
UEND 
(-ATCNO 
'1ATC^a 
i aTCno 
iATCND 
lATCN'D 
"iATCND 
i-»ATCKD 
'lATCNO 
MA ICh'D 
^ATC«^0 
IATCnO 
lATCNU 
-lAfCSD 
1A TCNO 
'lATCMD 
.^AICnD 
MA rc^D 
«» A rcNC 

rATCND 
'•ATC^O 
lAi C^D 

*. A T C \ D 
A f C V 0 
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120 J»LCS<n + L-l MATCVD 

130 IF<J.GE.LCS(I ).AND.j.LT.L:S( 1+1 ) ) L I ^= ( i I ) « Al { j ) "lAlC^-'D 

150 CONTINUE iATCNO 

200 WRITE(6»6100) i. lNt» T AdLt ( L ) -lAFCNJ 

800 COMTINUE lATC-^D 

900 COHTINUE MATC^D 

flATCMD 

C MATCNLf 

C GET CUT OF GRAPH MCDE MATLnD 

C foATC'^D 

WRITE(6»6400) HATCND 

lOOO CONTINUE 1ATCSD 

RETURN , MAKND 

C .-lATCND 

C FOR'^AT STaTEIESTS i^ATCND 

C UTCnD 

dOOO F0RHAr<l<^2{« ♦» k 1, » *» ) , 1 S<ILL ^ATC^O 

X 1X42(» IR MAICNJ 

X 1X42{* »#Rl#***)#lX*CArE&jRY*) ".AfCND 

6100 FUR»»AT{1X'.2(1XR2)#1i(Rb) MilC'^j 

6200 FORriATdH »13(10H 4) -^AIC^D 

6300 FCRMAT(*w») «ATC\0 

6400 F0RWA1(*R») tATCMD 

END "ATCnO 
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SUBROUTINE KCFREQ 

Jl!l^S!Mt^!;'.^I*^'-^'*'^^°'T*^'-^'CHOICE,FREU,PIR 
COHHON /GILO/ TITLE{8),T0DAY#NP6#TIM 
COMHON /GILl/ TASK(500),STA8LE( 16),A(8) 
riu^nt irl^li '<TASK,ISRT,NSV,N,NR,NKC,TID,CHOICE,NCOP 

♦SSrS^oof^^^^^^^ 

COMMON HAT (40000) 
CALL SHLSRT(TAaLE,NR#PTR) 
DO 100 L>1«NR 

IF(HO0(L#5O)-l.NE.O) GO TJ 50 
CALL TOP 
WRITE(6#6000) 
WRITE(6# 6010) 

WRITE 16. 6100) L, TABLE ( PTR{ L) ), FREQ(PTR(L)) 
CONTINUE 

FS521IlJ?ir*'^'Iv;J l^l*-*- ^'^^'0^ KNOwLEUt,fc CATEGORY*,/) 

F0RMAT(5X*L*, 8X*CATEGaRY*, 10X*FRtQUENC Y*, /# 

X5X*-* ,8X* ♦aOX* ♦,/) 

6100 F0RHAT<1XI5,8XR8,UXI5) 
RETURN 
END 



50 
100 
6000 
6010 



KCFREO 

EXTEND 

EXTE,"^D 

EXTEND 

EXTEND 

<CFREC 

KCFREQ 

-KCFREO 

KCFREO 

KCFREC 

KCFREO 

KCFREO 

KCFREO 

kCFKEu 

KCFREO 

KCFREO 

KCFREO 

KCFREC 

KCFKEO 
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6000 
6100 
6 200 



i«j8R0uriNc HEAD 

CjMf'GN /GILO/ TITLE^8>,T'"j:aY;.,PG, TI- 
C3>1>«3N /GILl/ TAiK(iOO),jrAdLt( 16>,A(6) 
Cu^^uN /GIL2/ NTAiK, IiRT,'>'iV,N.,.\R,NKC, |IO,CHciCE,.^.COf' 
CoJ•.»*u^ /GIL 3/ KC(500),Krf(500),TAdLE(b00),LC( 500), LC -S < PO 1) , N B { t»00 ) , 
♦ f-Kfcw( 500) IR ( 500 ) 
CQM:10n IAK 40000) 
*»RnE( 6,6000) 
»'<ITE(6#6100) 

WRITE(6,6200) TI fLE, fuOA-r, 1 1.^ 

WRITE( 6*6100) 

PETURN 

FJRMAU IHl, 10(/ IhO) ) 
FClR*AT(30X#i»( 10H»*»»»**» ) 

^GR^AT( /Iri 031 >4 A 1 0 , / 1 HO 3 Ix, 4A 1 0, / IriO, 4 9X A 10 , / Ih 0, 49 X Al 0, / ) 
t' 0 



lE Au 

ExTENiD 

C j^»'C^• 

EXTEND 

tXTENy 

E UtMO 

HEAD 

HEAD 

HEAD 

HEAD 

HEAD 

HEAD 

HEAD 

H: AO 

HEAD 



600 



iu8R0tri.>.t Tof- 

iNFcGE-^ TAJ^,^>rAdLE,A, TIJ, iA3LE,CHjlCE,rKti,^lK 

ci"^HuN /oiLO/ TITLE (a),Tu3Ar,sPG» ri^ 

C^r.ijs /GUI/ TAiK('jO^),>rAbLEllo),A((3) 

CL-'"'..f. /bl'.2/ f-TASK, n.R7,N;v,.<, N«,KKC, TID,CHjlC E, «(C'.^ 
Cu-^r^f /GIL 3/ ^ C ( 500 )» Kv ( 30 0) » T^JLb ( 500) »lC ( tJuG )» LC 5 ( b 01) »Nd (500) 
♦ FkfcJ(pOO)» Pf M -jOO) 
C>1.*»^JN AT(400UC) 
^(>G•^^■G^• 1 
CALL II-'ECri'') 

i»KlTE{o,600)TiUt»T .jAf. II^f^pG 
F'ji?«A|(lHl,2XbAiO,^(l;iAiO),^<*f'AGE«»I5,/) 



T 

C j^V'P'. 
C Ji-^N 

C 

c < T ENiD 
E<Tb'>JD 
Ex^E^[) 

r jp 

i 

I 

I i -^ 
f . 



26^ 



ERIC 



c 
c 
c 



SU3R.1uTlNt MOP 

INTB6ci( TASK»STAaLt»A,Tlu»l AiiLE#CHjIC6 »F^EO»PlK 

COMMON /GILO/ TIUE(3),l:>:)AY,,viPG#TIi« 

COM.MQN /GUI/ TASK(;)00),SrA3Lb(i6),A(8) 

COMMON /GU2/ NTASK,is^T,NW,N,Nfi,.-,KC,TIJ,CHuICe,sCOF 

COMMON MAFC^OOOO) 

NPG«NPG*1 

CALL TiMEdIrt) 

ENTER GRAPH MODE 

VRITE(6,600) 

WRITE (6#610)TITL6#T!.jDAr*TIf^#.^P6 
RETURN 

600 Fbi^MAfdriC) 

610 FOR^AKIH »2X8A10»2llXA10),2X*Pu3£»,l5,/) 
END 



Ci.'iMOV 

C Jn^CN 
EifTESO 
t< TEND 
EXTEND 
1'JP 
DP 
UP 
i IP 
JP 
1DP 



SuBRCLllNE SETRAY(A,M,8) sP7K-Af 
I»»Tt6fcK A, a i^w 

OIKEnSIjN a(n) <-t^w 
00 100 I.1,S .Av 

loo C3NTU0E %^ 

END '^'^^ 

i>':ri«Ar 



l>bBRCiuTlNE ERRORd) 

INTEGER TASK,STAbLE, A, TIO,TAdLt,ChuICt,hREw', PTk 

COMMON /GILv TITLt{8),T00At»MPG,Tn 

CjflMON /GUI/ . SK( j00),iTAbLc(16), A(6) 

CLMhON /GIL2/ M A SK» I SRT, nS o n»» N R/ ,\K C» T ID,CH CIC £, »JC1^ 

COHHIh .«iAT {<»0000) 

IFdiLE.O.DR.I.GT.DSTjP 

STOP 1 

END 



500) 



€ J ."'^ 0 Nj 
£XI£\(C 

C 

EKTENO 

EXTE^J 

ERR&R 

ERRCR 

EkROK 



ULOCK DATA 5LuCK 

INTEGER TASK, SI A6LE»A, TIO,T ABLE»CHUICE*FRcw„ r^TR C'lf-.^CN 

CJ^IMON /GILO/ TITLE (8),TuDA|'»NFG»TI'< CO'^-Cn 

CC^f^ON /GUI/ TASK(500),jrAaLi^(16),A(S) tXlEMO 

CuMMON /GIL2/ ^TAS^,ISRT» >iSV, <»-.R,%NC,riD,C,iuICE#N'CGP CUhrHs 
Cj-l^lCN /GIL3/ KC( 500),Kv(500),lAbLL{ 500J,LC(500),LCS(50i),Nb(300), EXTEND 

♦ FRE0(300),PTR( dOO) EXTEND 

CQMKQN MAT(40000) EXTEND 

DATA N>V/16/ iLJ.CK 

LATA ST ABLE/8RLuC'J.ioT'o oftCdJ ^AN.^, eR GOG-S T kG# 6 PHijr UTR, bLi,C'< 

X BkLEaUSHi P, 3Kjr(AL 1!SE,8RREaO ciE,b(<*RIT USE, oLiCK 

X 3R^■EIHJu^ »tRwOALITY »8RFIoLRAL » dRSy^SGL IC* riLLCK 

X fakTAXiJ'iQIC, dRIMPL ICITfdRF iNC £r<ri, bkHb^-.^ ERR/ 3LVCK 

t'^J 3LuCK 
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IDENf SHLSRU 



SHLSRT 



♦ 

♦ 
♦ 
♦ 

♦ 

* 

* 



CALL iHLSRK TAiil.£ , N ) 

CALL ShLSRK TABLt » N , PTk ) 



- aiS?.CT ( I\ PLACE ) ~,TRT 

- INJixECT >.jkT 



TABLE » ADDRtSi lF TABLt ( ARRAY 
N » M;«SEi< JF fcjROS IN FAbLc 

» AJD-^Eii OF AN! ARRAY TU Be J-jEO 
TABLE ( DIltNblijhtO PTR(N) ) 



A) A 



^•"■^ TED 
3 ]I»iTE^ 



IF THE THIRL AR^ij-idNT 
SCRf WILL 8E 
■ii Uili) AS A 
i-i^T biLL PE 
pf lM THE TH 



IS ABSENT THEN A 



DIRECT ( li pla; 
PERFuRMED, ELSE THE THIRD ARGJ''E^T VRL 
PlI'xTcK table ANj an i^.JlRECT { PjInTER ) 
PERFjKfEO. 3N EXIT THE -^flNTbR TABLE -ILL 
cLEltNTS '-f »TAaLt< l^ ASCENDI^w R3£R. 
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'^Hl SRT 
SHLSRT 
SHLSRT 
SHLSRT 
5riL*:RT 
SHLSRT 
SHL'^fT 
SHL'^^T 
SHLSRT 
'^HL'^RT 
SHLSRT 
SHL SRT 
SHL'^RT 
SHL'RT 
SHLST 
5HL<^RT 



iTBA 



t: >(TkT 


irlLSRT 


BSS 


2 




sr^A 


> AC 




?S 




JBI 


1 


;A2 


Al + dl 


iXfc 


AO 




A2 + t:l 



KE:>TORE AC 



5HL WT 
SHL^'T 
:HLSRT 
5HL^RT 
^HL '■R T 
SHLSRT 

SHL'^RT 
JHL'^RT 
•••HL'^^' T 
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SA2 


X2 








SHLSRT 


SAO 


Xl-1 


AO ■ FWA-1 






SHLSRT 


SA6 


STBA 


SAVE AO 






SHLSRT 


S37 


X2 








SHLSRT 


$36 


X2 


N - N (LENGTH) 






SHL'^RT 


N2 


X3#STBi 


IF AN INDIRECT SORT 


REQUESTED 




SHL.iRT 
SHLSRT 


SX6 


B6 


M ■ H/2 






SHLSRT 


AX6 


1 








SHLSRT 


SB6 


X6 








SHLSRT 


SB3 


Bl 


J « 1 






SHLSRT 


IR 


B6#STB0 


RETURN IF f» ■ 0 






SHLSRT 




B7-B6 


K ■ N~M 






SHLSRT 


SB2 


B3 


I ■ J 






SHLSRT 


SB!> 


B2^d6 


L « I + H 






SHLSRT 


SAl 


A0+B2 


Ad) 






SHLSRT 


SA2 


A0+B5 


A(L) 






SHLSRT 


1X4 


X2-X1 








SHLSRT 


PL 


X4»ST83 


IF A(L) .GT. A(I) 






SHLSRT 


BX6 


XI 








SHLSRT 


LX7 


X2 


INTERCHANGE A(L) ASO 


A(I) 




SHLSRT 


SA6 


A2 








SHLSRT 


SA7 


Al 








SHLSRT 


SB2 


B2-B6 


I ■ I-M 






SHLSRT 


GT 


B2iST62 


IF I .GT. 0 






SHLSRT 


SB3 


B3 + B1 


J » 4*1 






SHLSRT 


S32 


B3 


I - J 






SHLSRT 


LE 


63»B4,5TB2 


IF J .LE. K 






SHLSRT 


EO 


STBl 








SHLSRT 
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ST8! 




S82 fa6-Bl SHLfRT 

SAO X3~l AC t.- PW.4.-1 Oi^ '=0[NTcf' TABLE ^H' ^RT 

SA6 Abffji SAVe. 400R 'Jf A SHlSRT 

Jvt SHLSRT 
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SHLSRT 
SHLSi?T 

shl5;rt 

SHL SRT 

shl<;rt 

^ SHL5RT 



SHL'^-'. r 



SHL ^R f 
SHf.RT 
<hL?RT 

SHLSRT 

shl^^r r 
?hl'«;rt 

. _ ShL'JRT 



1X5 



S o XI 



If A c 
S A 7 



SHIS'? r 



SHL 

SHLSRT 



iA5 










A 5 8 1 






>32 


A 0 1 a i 


{ i ) 




^bi 








i AO 




R6o^irRE AO 




iXi 


Xi-1 






5A2 


S2 


CrtANGt P?: iNttK 


A R ;i f 1 




ft? ♦ ft 1 








X2~X1 






:A6 


A2 






IT 


32#83,iT8I5 






c ^ 








5nD 









c 

r 



ID' !5i ,• 
IOC cc^ri 



: L S r 

; H t ^ R T 

SHL^^ r 
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APPENDIX C 



A REVIEW OF THU HFALTH SERVICES 
MOIiiLITY STrDY METHODOLOGY * 



Some Nores on the Statistical Method iJrilized by the Health C-l 
Services Mobility Study. 

bv Eari E. Dr. vis 



An ?:va;uarion of the riealth Services Mobility Study Methodoiogv. C-29 
by Phillip R. Merrifield 



A Critical Review of the Mfithodology ind Statistical Treatment 
of Data in the Task Analysis and Career Ladder Design of the 
Health Services Mobilitv Scudv. 



by Mark I. Appelhaum 
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^ iTie three papf'zr^ p^'esented in this appendix were written in 19 76 
in response to requ'Sts from the Oirertor of H'^Mf, and its f»:ndin?' 
agency. The materi-ols reviewed did not iiK i'.iJe the full-soale 
application of HSMS me^hodology in dia^r-osti^ radiology and the 
tieveloprji-int of curricuiuTr. guidelines, which were reported iv. late 
19V6 .-^nd 1977 . 
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SOME NOTES ON THE STATISTICAL METHOD UTILIZED BY THE HEALTH 



SERVICES MOBILITY STUDY 



By Eirl E. Davis 



INTRODUCTION 



The purpose of this paper is to provide c» rationale foi the sta- 



tisLical procedures utilized by the Health Services Mobility Study (HSMS) 
in its task analysis method. Ti ^asic statistical technique used by HSMS 
to analyzt? its task data is that of factor analysis. HSMS use of factor 
analysis has been reported briefly in some of its documents [10, 11, 12, 
-3t. This paper proposes to provide a technical description and explana- 
tion of the techniques used aud statistical decisions made in connection 
with the HSMS method. We hope that this uill facilitate replication of 
the HSMS 'statistical anaiyr.es cnui will pruvid? a concrete basis in terms of 
which to discuss the rationale for the techniques adopted, to thus make 
possible critical review. 

We first present a brief description of the HSMS data base and 
the riSMS use of factor analysis.. Then some of the issues and somf of the 
literature of the field are discussed in connection with the particular 



This document was prepared by Dr. Davis in January, 1976, at the request 
cf the Health Services Mobility Study (HSMS). It haJ been edited by E'^eanor 
GilpatricK, Director of HSMS. Dr. Davis vas formerly Chief Consult:»^t of 
HSMS. rie IS currently Research Professor and Chairman of the Department of 
Social Psychology a'^d Sociology of The Economir and Social Research Insti- 
tute, Dublin, and Adjunct Professor, Department of Statistics, Trinity 
College, University of Dublin. 

Numbers in brackets refer to references listed at the end of chis p^^per. 
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procedures utilized in the HSMS method. The issues relate to (1) th^ ap- 
propriateness of HSMS data for factor analysis, i.e.» the basic assump- 
tions underlying the use cf parametric stati^stics in general, and factor 
analysis in particular, (2) the questi m of whether communality estimates 
or unities in the diagonal are preferable in che tactor analytic procedure, 
and (3^ the choice among different rotationc.1 procedures. These are consid- 
ered in the T'ght of HSMS objectives and the nature of the HSMS data. 

Terms 

A possible source of confusion in a discussion of factor analysis 
is various authors' use of terirs. For example, terms such as ^'principal 
coipponents," '^principal axes," "principal factors," "principal axis fac- 
tors,'' and others are used somewhat differently (and sometimes a bit 
Loosely) by different writers. In a recent criticism of an article- 
by Timmermans and Sternbach [45] one writer, reflecting Harman^-^ usage[16l, 
suggests that "the distinguishing haracteristics of factor analysis in 
contrasc to principal components analysis is the substitution of estimates 
of communal i ties in the diagonals in place of unities..." [36, p. 861]. 
Harman dis t inguishfiS between principal components and principal factors 
[16, p. 100] (i.e., component analysis anc' classical factor analysis [16, 
p. 346]). In this paper we use the term factor analysis t cover a variety 
of methods including the piincipal components method, unless distinguish- 
ing one method from anrth^'^r- 

We also prtifer to use the terr^ "principal axis (PA) factors" in- 
stead of "principal factors" to avoid confusion with the so-called "princi- 



pal-tactcr" technique, which is one of several faccor analytic techniques 
for exc-.^cting principal axis factors in order to reduce an m x m matrix 
to an iTi X k maLtiz of PA factors (k < m) . 

HSMS Data Base 

T'lis paper ascumes that the reader is acquaintev with the ^,en- 
eral features, objectives , and underlying data base use>^ in the HSMS method 
of task analysis as presented in relatively nuntochnical terms in various 
HSMS documents [10, 11, 12, 13]. 

The goals of the HSMS method are to desigu job ladders and cur- 
ri':ula based on the skill and knowledge requirements of tasks. This re- 
quires a descriptive use of tatistics. 

One feature cf the HSMS method is a carefully conceptualized def- 
inition ot the basic unit of observation, the task. The interrater reli- 
ability of the definition, usad to identify tasks in the field, was tested 
and showed ; satisfactory degree of reliability. 

The basic variables developed in the HSMS method ?re sixteen skill 
diwenslons and the knowledge categories of the HSMS Knowledge Classifica- 
tion System. These variables are applied to the task observations and are 
assigned values by use of the scales developed ^or each of the skills and 
a scale used for all the knowledge categories. Each oi' these seventeen 
scales was developed in a complex and car-^fully applied procedure using 
the Thurstone equal-interval scaling technique (9, AA], which gives these 
variables tne appropriate statistical properties. Each scale has its low- 
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est value at zero and its highest at 9.0. Some scales have as many as 
eight descriptors. No scale has less than five. 

The HSMS method incorporates preparation of the HSMS task data 
for analysis using the EDIT program designed for HSMS. Simple and "two- 
mode** factor analyses are then used to cluster the variables and tasks. 

In its initial preparation of the data, the EDIT program deletes 
variables from consideration that have a frequency across tasks below a 
selected minimum. This feature partly solves the problem of variables 
which take on a value of z^5ro for an excessive number of observations. 
It also permits the user to reduce the number of variables to an appropri- 
ate number ir relation to the number of observations, which is an impor- 
tant statistical consideration. The EDIT program is then used to "nor- 
malize" the data. That is, a nonlinear transformation is performed on 
the data to bring it into a closer approximation of linearity among vari- 
ables . 

HSMS Use of PCVARIM 



ERIC 



The initial factor analysis prograi.i used by HSMS for determining 
the nambe'* of factors in the solution and for clustering the variables 
has the name PCVARIM, which is an abbreviatior for Principal Components 
Factor Analysis with Varimax Rotation. This particular program has cer- 
tain features that incorporate decisions about some of the procedures 
subsumed under the general term 'factor analysis." The program uses a 
principal coiuponents technique ior arriving at pri)icipal axis factors 
("unities" in the diagonal rather than comraunalit/ estimates) , and it uses 
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orthogonal varimax rotations. The mat*-ix of variables is a correlation 
matrix rather than a covariance or a cross-products matrix. 

The PCVARIM program was originated at the University of Illinois. 
It was initially programmed by Paul Herzberg [17] under the supervision 
of Henry Kaiser in the late 1950 's. The factor analytic adaptation of 
the original principal components method consists essentially of an appli- 
cation of a form of triangular decomposition , which is described in sonc 
detail by Harman [16, pp. 101-103]. This is incorporated in a subroutine 
of the program referred to as TRIDI (a technique for tri-diagonal iz^?tion 
of matrices). Subseq uently, an extremely useful, widely used subroutine 
was developed by Householder, Ortega and Wilkinson, with the appropriate 
name of HOW. This subroutine involves a highly sophisticated set of pro- 
cedures for the numerical analysis of eigenstructures , and calls, in course, 
the other subroutines TRIDI, EIGVEC (eigenvector), and EIGVAL (eigenvalue). 
This subroucine was originally programmed by David W. Matula, under the 
direction of William Meredith at the University of California at Berkeley's 
Computation Center. (A more complete account of the mathematics of the 
subroutine HbW may be obtained from the chapters written by the originators 
of the techniques [see in 37].) 

The resulting PA factors obtained by PCVARIM are then rotated to 
simple structure using the varimax piocedure for orthogonal rotation de- 
veloped originally by Kaiser [20]. The history and mathematics of the de- 
velopment 01 this procedure for orthogonal rotation is described in corn- 
er 5 
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plete detail by Harman [16, Chapter 14]. We have come to call this set 

3 

of procedures "simple" factor analysis. 
HSMS Use of Two-Mode Factor Analvsis 



any set of data, simplr factor analysis is usually used to arrive at some 
structure and a parsimonious explanation of a set of variable s ,i,e,, ob- 
served variations (o\ variates) v^hich occur over a number of observed 
cases. In psychometrics , the cases, or observations, would usually be 
individuals In the HSMS research, the observations or sources of ob- 
served variations are the tasks involved. 

While there is interest in the factor structure of the vari- 
ables, i.e., skills and knowledge categories, the ultimate HSMS aim is to 
discover the underlying structure of the observations , i.e., ua3ks. The 
reason is that a f^oal of HSMS is the construction of job ladders based 
or. the tasks. other words, we wish to see how job tasks would clus- 

ter witn each other. In principle, ve could have used simple factor anal- 
ysi.- of the type described above, or any ^^ariation thereof, to cluster 
the tasks- The probler.. was that we then could not know what the bases 
were on which the tasks clustered. We wished to inspect the factor struc- 
ture of the skill and knowledge variables which comprise the HSMS data 
base as the basis for clustering the tasks. What was clearly needed was 

"Simple" factor analysis is described here in fairly general terms, since 
the sources lo which we refer contain the explicit mathematical formula- 
tions which permit complete replication. 



While, in principle, one can aprly simple factor anal>^is to 
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the capability of factoring in more than one mode simultaneously, in such 
a way as to establish a unique relationship among the multiple reference 
axes of the modes involved, i.e., variables and observations . 

The procedure used for clustering the HSMS task observations is 
a modified version of the Tucker-Mess.ick procedure for factoring an indi- 
vidual differences matrix [7, 49]. The program permits the extraction of 
principal axis factors for both observations and variables (two modes), 
based on a covariance or a correlation matrix of variables. It is then 
possible to rotate one mode to ''simple structure" by a varimax method, and 
to "counter-rotate" the second mode. For HSMS the first mode is the skills 
and knowledges variables; the second mod; is the task observations. Coun- 
ter rotation is done by obtaining the transformed characteristic vectors 
of the second (observations) mode induced by the varimax rotations of the 
first (variables) mode. [See 47 and 48 J We refer to this use of factor 
analysis as "two-mode factor analysis." 

A problem similar to the HSMS problem had been studied in psycho- 
metrics over some time, namely, how to establish "idealized subject types." 
In 1964, E. E. Davis and H. C. Triandis, working at the University of 
Illinois, read the seminal article by Tucker and Messick [49] describir^ 
a procedure for factoring an individual differences matrix, and were cware 
of the continuing efforts by Tucker in the direction of multi-mode factor 
analyses (e.g., 47, 48]. They were also quite fortunate, in having a good 
working relationship with Professor Tucker, who was in the same depart- 
ment at that time. They were thus able to "interrupt" Professor Tucker 
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in his concentration on perfecting three-mode (and n-mode) factor anal- 
ysis to apply the principles involved to two-mode factor analysis. 

Davis and Triandis were dealing with social attitudes and in- 
terpersonal perceptions, but the principle behind the technique utilized 

4 

is precisely what was called for to meet the aims of the KSMS method. 

In general terms, the two-mode factor analysis program performs 
the following functions. It permits the derivation of two principal axis 
factor matrices , one for variables (skills ind knowledges), and one for ob- 
servations (tasks), through a principal components technique, using tri- 
diagonalization, modeled on the PCVARIM program described above. The 
critical eltment, however, is the utilization of the Eckert-Young theorem 
[8] for approxima*"ing one matrix from another in such a way that it be- 
comes possible to rotate one matrix to simple structure and subsequently 
"counter-rotate" (to use a rather loose terminology) the other matrix in 
such a manner as to obtain the isomorphic relationship between the two 
sets of reference axes that is desired. 

In the case of the HSMS method, the first set of principal axis 
factors is rotated to simple structure by means of the varimax criterion. 
This is the first mode. Then the transformed characteristic vectors of 
the second mode are obtained by "counter- rotation" of the corresponding 



4 

Utilizing this technique, Davis and Triandis [7] were 'able to demonstrate 
ennpirically the validity of a model which provided the resolution of a long- 
standing conflict as to the determinants of social acceptance or rejection. 
This model and the studies and controversies leading up to its formulation 
and proof are perhaps best summarized in a replication study carried out 
by Goldstein and Davls[lAJ. 
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second mode vectors. The transformed characteristic vectors of the second 
mode are thus induced by the varimax rotation of the first mode. 

Either the observacion mode or the variable mode may be used as 
the first mode and the other as the second mode, depending on the nature 
of the data. Davis and Triandis obtained simple structure of the obser- 
vation mode first. In the case of HSMS, we obtain the simple structure 
of the variable mode first. The transformed characteristic vectors or 
factor loadings of the HSMS task observation mode correspono to, and thus 
can be interpreted in terms of, the corresponding skill and knowledge 
variable mode factors. The simple structure of variables reips to in- 
terpret the corresponding structure of tasks. 

5 

The matrix operations can be briefly conceptualized as follows: 
m = number of variables 
n = number of observations 
k = number of factors 

Y = raw data matrix with elements y^ ^ i = J,...,n; 
j = 1, . . . ,m 

X = rescaled matrix with elements x^^ ^ = (y. . -y.)/ (s.^TT) 

Wherp Sj = standard deviation of y,j« 

From the Eckert-Young theorem, we can say that: 

X U, * A, * 
n m n k k km 



In this presentation the observation mode is first rotated to simple struc- 
ture. In the case of HSMS application, the variable mode is first rotated 
to simple structure. 
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wheie 

U - raaracre: jfstic, Mf. tors of XX' 
V = characteristic, vectors of X'X 

\ = diagonal matrix of the square roots of the character- 
istic roots of both ^X' * X^ and ^X * X'^- 

= principal axis factors for observations. 
Now, let j^Tj^ be the transformation matrix such that 

n^k * k'^k = n^^k 

where R corresponds to the varitnax factors for observations. 
Then , 

n\ = m^k * k'^k * k'^ * k^'m 

" n^k * k'^""^k * k'^'m 
Let S = T""-*- * V' ; then, 

n^m = n^k * k^m 



varimax factor transformed charac- 
loadings for teristic vectors 

observations. for variables. 



The set of operations above is an abbi."eviated version of the en- 
tire matrix operations, designed primarily to illustrate the operation of 
the Eckert-Young theorem. Steps such as trie generation of the original 
and rescaled matrices and their conversion in.:o PA factor matrices are 
omitted. 

Davi:2 and Triandis originally used a covariancc ^Mtrix as the 

input to the factoring [7]. More recently, Ovv^/od and dissociates employed 

a cross-products matrix as the input to the niulti-moJe factor analysis 
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procedure (e.g. 33, 50]. Factor analysis is a generic technique ^.^lich c.in 
be applied to any type of matrix. Historically, factor analysis has used 
a matrix of correlations among all the variables of an original raw d<it<i 
matrix in the initial step. However, it is equally possible to convert 
the raw data matrix to a covariance matrix, n cross-products matrix, or 
some other form of matrix, and to factor the resulting transformed injtrix. 
The cross-products matrix is particularly applicable to the Osgood Seman- 
tic-Differential type of scales [33] because of their symmetry. We deiovnum-d 
that the HSMS data do not appear to lend themselves to the use of eitlier 
a cross-products or a covariance rratrix, and a conventional corrol.i- 
tion matrix is used, 

NOTES ON SOME STA TI STICAL ISSUES IN FACTOR ANALYSIS 

Aside : :"om the frequently heard (and cften q lite justifiei) crit- 
icism that factor analysis is used indiscriminately people who do not 
know what they are doing or why they are doing it, there are basic statis- 
tical problems involve^ with the use of parametric statistics general Iv 
ana factor analysis in particular. Obviously, any technique, no matlt-r 
what its intrinsic value, cai: be misused or used indi.'>crimin<ite 1 y . 

Controversies concerning the legitimacy of factor arialvsis ah a 
technique and concerning the best method of factor analysis characterized 
the development of the technique in the thirties and forties. Cureton de- 
scribes this in a rather witty and sarcastic way [6]. More recently, such 
authors as Harman [16], Cooley and Lohnes [5] and others have dealt with 
the subject. Harman states: 
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The many papers that appeared during the thirties 
and forties urging "this method" rather than "that 
method" had their piace in the growth of the sub- 
ject. However, wiLh a fuller understanding of 
the salient features of each method, and with the ^ 
increased efficiency of computations, the dif- 
ferences among the various methods no longer loom 
so ominously, and the followers of a particular 
approach are much more tolerant of the adherents 
''^ of an alternative scheme. [16, p. 10] 

Harman indicates that "the heated and inspired controversies 
about the 'best' method of factor analysis are over" [16, p. 9). As Cooley 
and Lohnes have put it, "only recently have students of factor analysis be- 
gun to see that the different procedures are suitable for different pur- 
poses..." [5, p. 129]. 

Two of the major statistical questions relating to factor anal- 
ysis as a technique have to do with the use of conanunality estimates or 
unities in the principal diagonal, and with procedures for rotation. Even more 
fundamental, however, are questions concerning the robustness of factor 
analysis. This section first discusses the issue of robustness, then the 
rationale for the use of unities in the diagonal of the correlation matrix 
as input to the factoring procedure, and, finally, the selection of a ro- 
tation technique in the HSMS application of factor analysis. 

The Data, Factor Analysis and Robustness 

In discussing preconditions for the use of factor analysis, 
Harman indicates that "all observed variables must be linearly related 
to one another" [16, p. 374]. He relaxes this requirement to include 
relationships that are monotonic. Another condition is that each observed 
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variable be normally distributed or at least not distinctly non-normal 
[16, p. 374]. 



In this section we first suggest that the variables involved in 



the HSMS method are more amenable to factor analysis than might be initial- 
ly assumed, and that it is appropriate to use factor analysis techniques in 
connection with them. Second, we suggest that modern statistical thinking 
and research findings indicate that factor analysis techniques are more 
robust than was previously thought to be the case, even with greater de- 
viations from normality or linearity than was previously thought tolerable. 



23] is that, when n becomes large enough, the deviations from t.\e "normal'* 

distribution become trivial and the application of parametric statistical 

analysis becomes justified, despite the reservations of earlier "pre-clas- 

sical" statisticians • The HSMS sample sizes are in practice large enough 

to meet the criterion. The n*s which have been encountered thus far have 

6 

been well over 200 task observations. 



The EDIT computer program, which is used to prepare the HSMS 
data for analysis, linearizes the data by a non-linear (logarithmic) trans- 
formation. In Kendall and Stuart's chapter on Canonical Variables [24, 



Sources in Kendall and Stuart [22, 23, 24] were referred to the present 
author by Professor Kendall in a discussion in which the legitimacy (from 
the viewpoint of a statistician) of applying a factor analytic model to 
the HSMS data base was discussed in some detail. Professor Maurice G. 
Kendall, Personal Communication, Dublin, October 22, 1973, 
Confirmation of the validity of using factor analysis with data of the 
HSMS type has also been derived in discussions of the present author with 
R* C. Geary, Dr, R. C, Geary, Personal Communication, June 17, 1975. 



A theme which runs through the work 



of Kendall 



and Stuart [22, 
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Chapter 43], the authors show that logarithmic transformation can be used 



to linearize otherwise non-linear data. The authors, in discussing trans- 
formations, state : 

Consider a transformation to new variables 5 given 
by 

4 = ax (43,4) 
where a is a matrix of coefficients. We confine our 
attention to linear transformations of this kind — 
non^linear situations are much more difficult to han- 
dle, and if they are suspected to exist an attempt 
should b2 made to linearize the data beforehand, for 
example, by a logarithmic transformation , 
[24, p, 286; emphasis added] 

Recently, Kruskal and Shepard pointed out (in a paper on "non-metric 
linear factor analysis" [28]) that, "the standard methods, though presuppos- 
incj; linearity, are generally quite robust in the face of both random error 
and all but the most severe monotone departures from linearity" [28, p. 
130]. These authors also point out the extreme expense (in computational 
time) of carrying out the rather strict procedure to achieve monotonicity 
and approximations to linearity which their program involves. They state 
that, "it is still doubtful whether there are any commonly occurring cir- 
cumstances in which it is worthwhile to resort to the much more costly 
computation required," and conclude, citing Shepard and Carroll [39], that: 

It now appears that, in order to achieve an exten- 
sion of (two-way) factor analysis of appreciable 
practical power, it may not be sufficient merely 
to weaken the metric assumption of linearity. It 
may be .lecessary to abandon even the assumption of 
monotonicity,' [28, p. 153] 

It would seem that, even with rather severe assumptions about 
departures from linearity in the data, the procedures which we have se- 
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lected for the HSMS method could not have led to any significant distor- 
tion of the results. Given the time and the cost of the computational 
factors involved in alternative techniques, the gain which could be ac- 
crued from the alternative techniques would not offset the costs. 



were the case, such departures would not significantly distort the results. 
Modern factor analytic theory suggests that the results to be gained by 
painstaking procedures to convert the data to follow the rigorous conven- 
tions of linearity do not yield results appreciably different from those 
obtained without transforming the data. 

With regard to distribution of the data, we find that the vari- 
ables which form the data base of the HSMS method do not grossly appear 
to depart from assumptions of "normal" distribution. Further, the require- 
ment of normality is relaxed when descriptive rather than inferential uses 
of statistics are involved. 

If we assumed that we were faced with the worst possible sit- 
uation, in which each of the HSMS variables has a value of zero for close 
to half the observations, and all other values are positive, non- imagi- 



nary, and definite, we would be facing a situation in which we were seek- 
ing to factor analyze dichotomized variables. This is the limiting case 
for the HSMS data developed thus far. 

For such a limiting .ase, Christof f ersson [4, p, 5] h« < shown 
that the use of factor analysis with dichotomized variables is entirely 



In other words, even if appreciable departures from linearity 
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feasible. He reviews the difficulty involved in factor analyzing a matrix 
of tetrachoric correlations, since such methods usually require (among 
other things) that the correlation matrix be Gramian (i.e., positive, semi- 
definite>and symmetrical , with unir.ies in the principal diagonal). He goes 
on to demonstrate how the basic factor analytic model can be adapted to 
dichotomized variables. 

Christof fersson uses two dilfferent maximum likelihood approaches 
vjhich take Into account variables that are dichotomized [4, p. 5], namely, 
(1) the conditional maximum likelihood method [31], and, (2) the uncon- 
ditional maximum likelihood method [3]. The latter is an extension of 
the normal factor analytic model based on an estimation of parameters of 
the basic factor model. It uses the generalized least squares principle 

(the GLS-Estimator) , and involves the tetrachoric expansion put forth by 
7 

Kendall [21]. 

Whelan [51] uses a Monte Carlo approach to an examination of 
8 

factor analysis. His findings appear to verify those of Christof fersson 

concerning the possibility of factor analyzing dichotomous variables. We 

may also infer the acceptability of the HSMS data from the following: 

^ Christof fersson and his associates Anderson and Muthen [1] report that 
they have developed a computer program for carrying out factor analysis 
of dichotomized variables, which is available on request. They point 
out that, so far, they have run into difficulties with computational time. 
It is probably true to say that the two-mode program used by HSMS is; one 
of the few computer programs available which not only carries out a fac- 
tor analysis meeting the needs of HSMS, but also is within the bounds of 
reasonable ccnputer costs. 

^ Monte Carlo approaches have been used extensively in recent years in non- 
metric scaling (e.g. Klahr [25], Sherman [AO]), as well as in the investi- 
gations of metric problems such as those posed by factor analysis. 
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Quite frequently in economic and social research 
important variables can only be measured at a 
dichotomous or polychotomous level. The inclu- 
sion of such variables violates the assumption 
of normally distributed, continuous variables on 
which factor analysis is based, A related, though 
less serious, difficulty arises when variables 
are measured on a scale containing only three, 
five or seven points. The larger the number of 
points on such a scale uhe more closely the vari- 
ables conform to the assumption of continuity. 
Any number of points greater than four is gen- 
erally assumed to provide a sufficiently close 
approximation to continuity for the purposes of 
factor analysis, [51, p, 16] 

To investigate the problem of dichotomous variables Whelan took 
the data which he had earlier analyzed using several factor analytic pro- 
grams and transformed them into dichotomized variables by setting each 
negative value equal to -1/2, and each positive value eaual to +1/2. This 
set of dichotomized data was then factor analyzed by means of one of the 
standard programs. 

The author concludes that: 



It may be seen that the actual and estimated struc- 
tures are quite similar, the highest correlation 
observed between estimates and actual being over 
0,9,,,, The relatively small deterioration in the 
quality of the estimat'is obta'^ned when dichotomized 
data are used is therefore quite striking. [51, pp 
16-17] 



In a further analysis of the data, following an even greater as- 
ymptotic transformation in the distribution of the data, Whelan tested out 
a hypothesis put forth by Raven, Ritchie, and Baxter [38], suggesting 
that factor analytic results may be an artifact of the factor analysis 
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algorithm in those cases where high proportions of the sample tend to 
have either very low or very high scores • This is analogous to the ex- 
treme case with HSMS data where there are "coo many zeros," 

In order to test the hypothesis, Whelan assigned artificially 
extreme values to the set of variables on which he had a known factor struc- 
ture and found "no distinct pattern in the loadings •" He concluded that 
the hypothesis found no confirmation, Whelan says that, "we see that even 
in the case of unequal endorsement rates [scores], the estimates of the 
factors derived by the programme are still quite good, since all the cor- 
relation coefficients are greater than 0,75*' [51, p. 18]. 

Communality Estimates or Unities in the Diagonal 

In describing the distinction between the principal components 
method and other factor analytic techniques, Harman refers to R as a ma- 
trix of observed correlations among variables. He goes on to state: 



A set of n variables can be analyzed either (a) in 
terms of common factors only, by inserting unities 
in the diagonal of R; or (b) in terms of common and 
unique factors, by inserting communalities in the 
diagonal of R, These two approaches, of course, 
corre'^pond to the component analysis and the clas- 
sical factor analysis models, respectively •••• In 
the first instance R is a Gramian matrix, generally 
of rank ja, and the factor solution 

(16.1) z = Af 

is in terms of n common factors. Since A is a square 
non-singular matrix, in this instance, it will have 
an inverse. Then the required factor measurements 
are given simply by: 
-I 

(16.2) f = A z. 

This solution is determined exactly, is unique, and 
involves no "estimation," 
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However, when the factor model involves common 
and unique factors the solution is not so 
Simple, Then the total number of factors ex- 
ceeds the number of variables, and an inverse 
does not exist for the factor matrix M. The 
generally accepted procedure, in this case, 
is to resort to the "best fit" in the least 
squares sense [i.e., communality estirr^tos"! . 
[16, p. 346] 

In an early paper [35] Karl Pearson set forth the "method of 
principal axes" which has formed the basis for the method of principal 
components • The method of principal components outlined by Pearson is 
na longer used in its original form, but rather in the form of spe- 
cific adaptations to factor analysis of this technique, such as those made 
by Hotelling [19] and later scholars. 

Cooley and Lohnes state that one of the major uses of factor 
analysis is "to find ways of identifying fundamental and meaningful dimen- 
sions of a multi-variate domain." They then go on to say that: 

this "ccnstruct-seeking" task of factor analysis 
is most frequently accomplished today by first 
conducting a principal-components analysis, and 
by then U5;ing the resulting principal factors as 
a set of reference axes for determining the sim- 
plest structure, or most easily interpretable 
set of factors, for the domain in question. [5, 
p, 131] 

Whether one uses a principal components method with unities in 
the diagonal or other factor analytic techniques with communality esti- 
mates in the diagonal is actually a choice regarding the extraction of 
variance or reproduction of the observed correlations. According to Harman: 

...An important property of [the principal com- 
ponents] method, insofar as the summarization of 
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data is concerned, is tnat each component, in 
turn, makes a maximum contribution to the sum 
of the variances of the n variables. For a prac- 
tical problem only a few components may be re- 
tained, especially if they account for a large 
percentage of the total variance. Hov/ever, all 
the components are required to reproduce the cor- 
relations among the variables. 

In contrast to the maximum variance approach, 
the classical fac t or analysis model is designed 
to maximally reproduce the correlations. • 

...each of the n observed variables is described 
linearly in terms of m (usually much smaller than 
n) common factors and a unique factor . The com- 
mon factors account for the correlations among 
the variables, while each unique factor accounts 
for the remaining variance (including error) of 
that variable. [16, p.. 15] 

The principal components approach was selected for the HSMS data 
since it appears to be a logical use for an essentially descriptive under- 
taking. However, we did carry out an analysis to see whether the use of 
communality estimates in the diagonal would make an appreciable differ- 
ence. 

A number of methods have been proposed for estimating communal- 
ities. As Harman states, "As a matter of fact none of the methods has 
been demonstrated to lead to minimal rank of the correlation matrix" 
[16, p. 83]. Harman goes on to say that: 



As a saving grace, there is much evidence in the 
literature that for all but very small sets of 
variables, the resisting factorial solutions^ are 
little affected by the particular choice of "com- 
munalities" in the principal diagonal of the cor- 
relation matrix. [16, p. 83] 
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Before collecting main test data we compared factor solutions 
on pretest data obtained by the PCVARIM principal components technique 
(including use of unities in the diagonal) with ones resulting from squared 
multiple correlations as cottimunality estimates in the diagonal and itera- 
tion by refactoring. (We used the BMD package from UCLA which has this op- 
tion in it.) We then systematically compared the resulting factor struc- 
tures by means of the coefficient of congruence developed by Tucker (461 
and Wrigley and Neuheus (521. In comparisons between these two methods, in- 
volving two different sets of data, we consistently obtained coefficients 
of congruence well in the .90's, and concluded that it made no significnnl 
difference which method was used with the HSMS data. 

In a later run with 273 observations and 144 variables we compared 
our PCVARIM 6-factor solution (selected after inspecting all solutions 
from two factors to ten factors) with the BMD 6-factor solution. We again 
found great similarities in the loadin£,s. 

These results supported our decision to stay with the type of anal- 
ysis which we originally selected as best for our needs, namely, two-mode 
factor analysis, which in the program available to us at the time incor- 
porated a form of principal components technique. 

The Question of Rotational Techniqu e 

Aiiother question which has occupied the attention of factor ana- 
lysts concerns techniques for rotating the initial principal axis factors- 
The PA factors, extracted by whatever technique, are initially unrotated 
and usually not very interpretable in their original form. Subsequently, 
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any one of several rotational techniques is applied in order to achieve 
"simple structure" [A2] in the form of a set of rotated factors which 
arc generally more interpretable. A variety of techniques ranging from 
the early hand-rotational techniques of Thurstone [43] to later develop- 
ments involve various analytical solutions for orthogonal or oblique ro- 
tations . 

As mentioned earlier, we chose to use orthogonal rotation with 
our data, using Kaiser's varimax criterion [20]. We chose a solution 
which would not have correlated factors because the ultimate objective is 
to separate tasks into separate job ladders. 

Since there has been a great deal of discussion about the var- 
ious advantages and disadvantages of orthogonal versus oblique rotations, 
we decided early in the selection process for the HSMS method to conduct 
an empirical test as to whether the particular method of rotation made 
any significant difference with our test data. We used the BMD factor 
analysis program which contains a variety of options. We subjected sets 
of pretest data to both varimax orthogonal rotations and oblique rotations 
of the oblimax and oblimin types by varying the magnitude of beta in the 
basic rotational equation. We again used analytical comparisons involving 
coefficients of congrusnce, as described above. We consistently found the 
comparisons between the factor solutions yielding coefficients well into 
the .90's. 

It may well be the case that for certain psychological variables 
different factor solutions are obtained depending upon whether one decides 
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on this or that orthogonal or oblique rotational method. Our empirical 
eviJence suggested, however, that with the HSMS data, no significant dif- 
ference is found when one changes from one technique to the other. This 
suggestion of the basically orthogonal factor structure underlying the 
HSMS data and the robustness of factor analytic techniques applied to them 
seemed to support our choice of the two-mode program and the decisions it 
implies. 

Conclusion 

The HSMS method has been carefully developed from a statistical 
point of view. Decisions made at every step along the way were made con- 
sciously. Review of the more recent factor analytic literature suggests 
that our original decisions were taken on sound grounds and tend to cor- 
roborate the initial decisions which we made. 



REFERENCES 



1. Andersson, C.G., Christof f ersson , A., and Muthen, B. FADIV; A Com- 

puter Program for Factor Analysis of Dichotomized Variables . Re- 
search Report 74-1, Uppsala University, Statistics Department » 
1974. 

2. Barlow, R.E., Bartholomew, D.J., Bremner, J.M., and Brunk, H.D. 

Statistical Inference Under Order Restrictions . New York: John 
Wiley and Sons, Inc., 1972. 

3. Bock, R.D., and Lieberman, M. Fitting a response model for n dichot- 

omously scored items. Psychometrika , 35, 1970, 179-197. 

4. Christof ferssoa, A. Factor analysis of dichotomized variables. 

Psychometrika , 40, 1975, 5-32. 

5. Cooley, William W., and Lohnes , Paul R. Multivariate Data Analysis , 

New York: John Wiley and Sons, Inc., 1971. 

6. Cureton, Edward E. The principal compulsions of factor-analysis. 

Harvard Educational Review , 9, 1939, 287-95. 

7. Davis, E.E., and Triandis, H.C. An Exploratory Study of Intercultural 

Negotiations . Technical Report //26, ONR Contract #177-472, Nonr- 
1834 (36). Urbana: University of Illinois, 1965. 

8. Eckart, C, and Young, G. The approximation of one matrix by another 

of lower rank. Psychometrika , 1^, 1936, 211-18. 

9. Edwards, A. L. Technique s of Attitude Scale Construction . New York: 

Appleton-Centnry-Crof LS, 1957 . 

10. Gilpatrick, Eleanor. Final Report for the Period October, 1 967 Through 

March, 1972 . Technical Report No. 11. New York: Health Services 
Mobility Study, 1972. 

11. Gilpatrick, Eleanor. Suggestions For Job and Curriculum Ladders in 

Health Center Ambulatory Care: A Pil^^t Test of the Health Services 
Mobility Study Methodology . Research Report Nos. 4 and 5. New York: 
Health Services Mobility Study, 1972. 

12. Gilpatrick, Eleanor. First Progress Report For Phase Four: April ly 

1972 to March 15, 1973 , Technical Report No. 12. New York: Health 
Servicjis Mobility Study, 1973. 

13. Gilpatrick, Eleanor. An Introduction To The Work of The Healt h Services 

Mobility Study as ^: April, 1975 . Technical Report No. 13. New York: 
Health Services Mobility Study, 1976. 

G-24 



297 



14. Goldstein, M* , and Davis, E.E. Race and belief: A further analysis 

of the social determinants of behavioral intentions. Journal of 
^ Personality and Social Psychology , 22, 1972, 346-355. 

15. Harman, Harry H. Factor analysis. In H.S. Wilf and A, Ralston 

(Eds.), Mathematical Methods for Digital Computers . New York: John 
Wiley and Sons, Inc., 1960, 204-12. 

16. Harman, Harry, H, Modem Factor Analysis . (2nd' Ed., Rev^) Chicago: 

University of Chicago Press, 1967. 

17. Herzberg, P. A, The Parameters of Cross-Validation . Urbana: Univer- 

sity of Illinois, Department of Psychology, 1967. 

18* Hoeffding, W. The large-sample power of tests based on permutations 
of observa^tions. Annals of Mathematical Statistics , 23, 1952, 169. 

19. Hotelling, Harold. Analysis of a complex of statistical variables 

into principal components. Journal of Educational Psychology , 24 , 
1933, 417-41, 498-520. 

20. Kaiser, H.F. The varimax criterion for analytic rotation in factor 

analysis. Psychometrika , 23 , 1958, 187-200. 

21. Kendall, M.G. Relations connected with the tetrachoric series and 

its generalisations. Biometrika , 32, 1941, 196. 

22. Kendall, M.G., and Stuart, A. The Advanced Theory of Statistics . 

Vol. I (3rd. Ed.) London: Charles Griffin and Co.^Ltd., 1969. 

23. Kendall, M.G., and Stuart, A. The Advanced Theory of Statistics . 

Vol. II (2nd. Ed.) London: Charles Griffin and Co., Ltd., 1967. 

24. Kendall, M.G., and Stuart, A. The Advanced Theory of Statistics . 

Vol. Ill (1st. Ed.) London: Charles Griffin and Co.»Ltd., 1966. 

25. Klahr, D. A Monte Carlo investigation of the statistical significance 

of Kruskal's nonmetric scaling procedure. Psychometrika , 34, 1969, 
319-330. 

26. Kruskal, J.B. Nonmetric multidimensional scaling: A numerical method. 

Psychometrika , 29, 1964, 28-42. 

27. Kruskal, J.B. Monotone regression-- Continuity and differentiability 

properties. Psychometrika , 36 , 1971, 57-62. 

28. Kruskal, J.B., and Shepard, R.N. A nonmetric variety of linear factor 

analysis. Psy chome t rika , 39 , 1974, 123-157. 



G-25 

293 



29. Ledennann, W* On the rank of the reduced correlational matrix in 
multiple factor analysis, Psychometrika , 2y 1937, 85, 

30* Lingoes, JX*, and Guttman, L, Nonmetric factor analysis: A rank 
reducing alternative to linear factor analysis. Multivariate Be- 
havioral Research , 1, 1967, 485-505, 

31* Lord, F.M. An analysis of the verbal scholastic aptitude test using 
BjLrnbaum's three-parameter logistic model. Educational and Psycho- 
logical Measurement , 28, 1968, 989-1020, 

32* McDonald, R,P. A general approach to nonlinear factor analysis. 
Psychometrika , 27, 1962, 397-415, 

33, Osgood, C.E, Exploration in semantic space: A personal diary. The 

Journal of Social Studies , _27, 1971, 5-64, 

34, Osgood, C.E,, Suci, G,J,,and Tannenbaum, P,H, The Measurement of 

Meaning , Urbana: University of Illinois Press, 1957, 

35, Pearson, K, On lines and planes of closest fit to systems of points 

in space. Philosophy Magazine , 6^, 1901, 559-72, 

36, Posavac, E,J, Analysis of human chronic pain. Science, 187 , 1975, 

860-61, 

37, Ralston, A,, and Wilf, H.S. (Eds,), Mathematical Methods for Digital 

Computers , Vol, II, New York: John Wiley and Sons, Inc, 1967, 

38, Raven, J,, Ritchie, J., and Baxter, D, Factor analysis and cluster 

analysis. Economic and Social Review , 2y 1971, 

39, Shepard, R,N,, and Carroll, J,D, Parametric representation of non- 

linear data structures. In P,R, Krishnaiah (Ed,), Multivariate 
Analysis , New York: Academic Press, 1966, 561-592, 

40, Sherman, C,R, Nonmetric Multidimensional Scaling: An Empirical and 

Theoretical Investigation , Unpublished Ph,D, dissertation, Toronto: 
University of Toronto, 1970, 

41, Spearman, Charles, General intelligence, objectively determined and 

measured, American Journal of Psychology , 1^, 1904, 201-93, 

42, Thurstone, L,L, Multiple factor analysis. Psychological Review , 38, 

1931, 406-27, 

43, Thurstone, L,L, Multiple Factor Analys is, Chicago: University of 

Chicago Press, 1947, 

44, Thurstone, L,L, , and Chave, E.J, The Measurement of Attitude , Chicago 

University of Chicago Press, 1929, 

0-26 



ERLC 



2.9,9 



Timmermans, G. , and Sternbach, R.A. Analysis of human chronic pain. 
Science , 184 , 1974, 806, 

Tucker, L,R. A method for synthesis of factor analysis studies. 
Personnel Research Section Report No, 984 , Washington, D.C: De- 
partment of the Army, 1951. 

Tucker, L.R, The extension of factor analysis to three-dimensional 
matrices. In Fredericksen (Ed,), Contributions to Mathematical 
Psychology , New York: Holt, Rinehart, and Winston, 1964. 

Tucker, L.R, Some mathematical notes on three-mode factor analysis. 
Psychometrika , 31, 1966, 279-311, 

Tucker, L.R,, and Messick, S. An individual differences model for 
multidimensional scaling. Psychometrika , 28, 1963, 333-367. 

Tzeng, 0, Differentiation of Affective and Denotative Meaning Systems 
in Personality Ratings Via Three-Mode Factor Analysis . Unpublished 
Ph.D. dissertation. Urbana: University of Illinois, 1972. 

Whelan, B.J. Factor fiction: A Monte Carlo approach to factor analysis 
Seminar paper delivered at The Economic and Social Research Institute 
Dublin, June, 1975. 

Wrigley, C, and Neuhaut , J. A. The matching of two sets of factors. 
Contract temorandum Report , A. 32, Urbana: University of Illinois, 



1955, 



G-27 




AN EVALUATION OF 

1 

THE HEALTH SERVICES MOBILITY STUDY METHODOLOGY 
By Philip R. Merrifield^ 



INTRODUCTION 



This report is presented in four major sections, each of which 
contains a brief summary and an evaluation, with primary emphasis on meth- 
odological issues • It is unavoidable that certain substantive issues 
arise; it is hoped that any ignorance on this evaluator^s part of the 
complex area of health services may be forgiven, and that such deficien- 
cies have small, if any, effect on the methodological coniments. The four 
sections are: 



Goals: importance; specificity; attainability. 

Strategy: appropriateness for stated goals; applica- 
bility given presumed resources; awareness of options; 
inferred familiarity with tools and methods proposed • 

Performance: analysis of career and educational ladders 
and lattices; task identification; skill and knowledge 
identification; scaling; reliability and validity of 
scales and derived measures; interrelations among tasks, 
skills, and aspects of knowledge; documentation • 

Overall summary and suggestions. 



This document was completed by Dr, Merrifield in July, 1976, and is in re- 
sponse to a request for review of the Health Services Mobility Study (HSMS) 
method, Documr^nts supplied for review purposes included the Davis paper 
[1] (which precedes this), listings of the HSMS computer programs request- 
ed by the author, early HSMS documents not currently available [2], Re- 
search Report Numbers A and 5 [3], Technical Report No, 13 [4], and cur- 
rent HSMS scales not yet published, Dr, Merrifield also discussed as- 
pects of the methodology with Eleanor Gilpatrick, Director of HSMS, 
Numbers in brackets refer to references listed at the end of this paper, 

Dr, Merrifield is Professor of Educational Psychology at New York Univer- 
sity and is engaged in a number of ongoing research and evaluation pro- 
jects in the behavioral sciences and on learning. He is also an educa- 
tional consultant, 
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GOALS 



On first encounter, the goals of this project are awesome. To 
bound, dimenslonalixe, and map sur'i an area — not only topo logically but 
^ with a great dea of quantification — is surely an effort worthy of a 

great explorer. But affect aside, the importance of the effort is clearly 
and convincingly stated. Obviously, the work needed doing. It is to be 
hoped that the results will not now be set aside due to lack of funds dur- 
ing the present crisis. Decision-makers must be made forcefully aware of 
Its value as well as in the domain of educational and career development 
that it directly concerns. 

I note with approval the level of specificity and objectivity in 
the statements of goals and of anticipated end results from the project 
effort. It is a measure of confidence and competence that such specific- 
ity is introduced early in the documentation, for in so doing one runs 
great risk from later evaluators. A vague objective can be interpreted 
favorably or unfavorably: in these reports there are definite commitments, 
which I applaud. I shall attempt to make my comments equally specific, 
both as to apparent difficulties, successes, and suggested actions. 

With all their importance and specificity, are these goals at- 
tainable? Given the state of the art of job analysis and psychological 
measurement, is it possible to prepare the extensive materials needed for 
such a broad program? In this domain, new knowledge and new tools de- 
velop rapidly, and a certain tolerance for Sisyphean labor must char* 

acterize those leading the project. I think it a decision wisely made 
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to begin with a prototype system, with the aim of verifying strategy and 
method rather than obtaining new substantive knowledge. As a spin-off, 
however, it would appear that some very interesting results were obtained, 
and might provide a basis for planning the curriculum-building and selec- 
tion-placement aspects of the general problem. In reporting results of 
the HSMS pilot test [3], Gilpatrick quite properly suggests a modicum of 
caution in generalizing from these results; although the outcomes are 
satisfying in terms of what was expecLeu, the limitation of the pilot 
study to a single institution and the relatively small ratio of replica- 
tions to variables warrant a concern for further data. 

With regard to Goals, then, it appears to me that the staff of 
this project knew in quite specific terms where it wanted to go, knew 
where their objectives fit into the larger scheme, and made very reason- 
able initiating decisions to reach those goals. 

STRATEGY 

The basic scheme for the collection of data seems to involve 
three major sources of information: jobs, as carried out by performers; 
tasks, as components of jo>)S; and requisite skills and knowledge, pre- 
sumed to have been already developed and/or learned by incumbent per- 
formers of jobs. Obviously, the skills and knowledge dimensions could 
form a framework for efficient training for new employees. Similarities 
of tasks across job categories would allow for horizontal transfers of 
performers from one job ladder to another, withouc the need to re-estab- 
lish their possession of skills and knowledge in the new job setting. 
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Some orientation in the new setting might be lequired to maximize the 
transfer of skills to the new job. 

It was believed that a system could be developed to "score" each 
task on each of the skills and knowledge components; from this score ma- 
trix, the technique of factor analysis could disclose groupings of skill 
and knowledge components (hereinafter SK) which would be conceptually 
simpler and perhaps highly related in action. For example, if two skills 
were both required at a higher level by one subset of tasks than they v^ere 
required by the subset of the remaining tasks, the two skills would be 
correlated and would "load" on the same factor; the factor, in turn, would 
be defined by the skill and knowledge components that loaded on it. If 
the number of groupings of SK variables is suitable for describing the 
differences and similarities among the tasks, then factor analysis in the 
classic factor-score model or in what Davis [1] presents as two-mode fac- 
tor analysis may be used. 

At this point, a trade-off that may be important occurs. In or- 
der to use two-mode factor analysis, it is necessary to compute factors 
from the cross-products (or covariance, or correlations) among tasks, as 
well as among SK variables. This computation requires that each task be 
represented only once, to permit the computation of the inverse of the 
appropriate matrix. However, if a number of tasks appear cori»mon to a ^ 
number of jobs, should they not be represented more frequently in the 
total space being analyzed? It seems to me that concerns for representa- 
tiveness would indicate such inclusion, which might well change the values 
of the correlations among SK variables and thus have some effect on the 
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factors. I should like to recommend some exploration of this possibil- 
ity, and assert that the factor-score model would handle ':ha problem 
3 

quite well. 

In this connection, I must point out that the Davis paper is not 
sufficiently explicit regarding the source of data for the two-mode anal- 
ysis: specifically, the computation of the core matrix should be Includ- 
ed. My reading of the program by which the data reported were obtained 
assures me that the procedures themselves were sound, but the reporting is 
incomplete, at least for unsophisticated users. 

The question of using correlations or cross-products requires 
more discussion, especially concerning the information about differences 
between means of tasks that is lost when correlations are 'ised in two- 
mode factor analysis. The existing program provides options that could 
be explored in future data analyses, although it is appropriate to take 
the procedure used in this repo..r as a first priority. 

As for other strategic decisions regarding the factor analysis, 
ray preference would be for principal factors, with communalities in the 
diagonal cells, rather than principal co-iponents. In the latter method, 
the number of factors is sometimes too large and sometimes too small, 
depending on the magnitude and pattern of the correlations. However, 



3 

A factor score is defined as the sum of products of factor loadings of 
variables and standard scores of the task on the variables, weighted by the 
inverse of the correlation matrix. 
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considering the large number of variables per factor in the present Ffdy, 
this concern is more a matter of preference than of criticism. It would 
be interesting to see the congruence coefficients for the factor matrices 
resulting from different combinations of correlations, cross-products, 
principal components and principal factors. 

To ask a specific question of the data, is the correlation between 
the knowledge categories "Drug excretion" and "Biochemistry of nutrients" 
[reported in 3, p. 3-10, Figure 7] actually near .80? One can estimate 
this by multiplying their loadings on Factor I (.92 and .86). Is there 
really that much consistency between tasks requiring both kinds of knowl- 
edge (or not requiring either)? If the correlation, in fact, is not near 
the value computed from the loadings on a factor, then some further ex- 
planation is due. One possibility is that the high loadings are artifacts, 

resulting from the use of 1.00 rather than comDunality in the diagonal 
4 

cells . 

In the footnote to Figure 7^ the wording is more dramatic than 
warranted ("partake," and "tend to rise in an interrelated manner" are 
particularly jargon-y) . A factor has no life of its own, nor does a vari- 
able. A factor is a collection of variables with regard to which the tasks 
are differentiated in the same — or nearly the same way. For example, with 

^ Editor's note: The actual correlation coefficient is .94. 

^ Footnote reads as follows: "Note: Factor loadings represent the degree 
to which skill and knowledge variables partake of the factor. Loadings 
are standardized and range from +.00 to +.99. Loadings of .41 or higher 
are shown. These variables tend to rise in an interrelated manner." 
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regard to Factor I reported in Figure 7 [3, pp. 3-10 through 3-14], the 
272 tasks are differentiated from each other very strongly by the amount 
of information they require about drugs, and to a less€*. extent by the 
amount of information they require about physiological and anatomical 
system?. The variables do not "rise and fall together." They are some- 
what alike in the ways they serve to differentiate one task from another, 
in this specific set of tasks. 

Considering the obvious relations among skills and the probable 
relations among aspects of knowledge, the choice of orthogonal over oblique 
rotation seems open to discussion. While the desirability of simple struc- 
ture need not be re-emphasized, we should remember that Thurstone's search 
for simple structure among the many factors in the domain of aptitudes and 
achievement led him to invent oblique rotation. Here arises another trade- 
off. If the purpose is to define job ladders so that they are as unlike 
as possible, with a minimum of lattice-relations, then orthogonal rota- 
tion is more appropriate; if one wishes to maximize the mobility by search- 
ing for as many lattice-relations as possible, then oblique rotation, 
especially if tasks are replicated in some representative fashion, as 
referred to earlier herein, would be more useful. At the present time, 
there are no technical restraints on the choice of rotational option; when 
the decision was made, computing capability was much less flexible than 
currently. 

Regarding the determination of jobs as clusters of tasks, the 
grouping procedure based on loadings that is used in this report is logi- 
cally sound. Again, so rapidly does technology grow, statistically ori- 
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ented clustering techniques are now available which would simplify the 
work but probably not change the outcome. A further elaboration of the 
SK-task-job relation could be evolved using multiple-regression models • 
Ratings could be made of the involvement of each SK factor in each job; 
these ratings could be used as weights and multipled by the task-factor 
scores (or loadings) to develop a task-job index. The foregoing sugges- 
tion is based on my interest and should not be interpreted as critical 
of the procedure used in the study being reviewed. 

PERFORMANCE 

The selection of putative skills and areas of knowledge seem to 
me to be consistent with the job analysis literature. With regard to 
skills, Guilford's tri-partite model waw referred to but not adopted com- 
pletely, so that one reads of figural skills and classification skills 
which represent a rather large collection of Guilford's hypothesized fac- 
tors. In this instance, however, the reduction in the set of constructs 
has been made on the basis of expert judgment in the health field and is 
thus defensible; one should expect, however, that the skills factors might 
be more related here than in Guilford's model. 

Specifications for both domains, skills and knowledge, are usable 
and relevant. The coding system for knowledge is interesting and will 
no doubt have further payoff as curricula are developed. 

The choice to use equal interval scales is sound. There is, how- 
ever, a substantial difference between equal-appearing scales based on 
judgments and equal-interval scales resulting from intensive analysis of 
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winy specific judgments. The choice to use equal-appearing intervals in 
this study seems to put a large burden on whether judges can follow the 
plea to assign values at equal intervals. It is not clear from the re- 
port whether judges were given any orientation regarding the difference 
between numerical intervals and psychological distances. In contrast, 
Thurstone's simplest empirical method provides for computing the average 
of the standard deviations for two adjacent items: this average becomes 
the unit of distance between those items, and similarly for other pairs of 
Items. The mean or median of the item serves to determine its place in 
the sequence. 

For example* let us use the data presented in Gilpatrick's Table 
1 (reproduced herewith). Table 2, below, presents new scale values arrived 
at for two of the scales using the following method: we may use the medi- 
ans to array the items, and the interquartile range as an estimate of the 
standard deviation (within a constant); if we assume a normal distribution 
of judgments, and look for an overlap of distributions of abou: 10% between 
adjacent items, it turns out that we can use the sum of the two interquar- 
tile ranges (twice the average) as the distance between adjacent items. 

In Table 2, the scale values are, of course, in the same sequence. 
For "Guiding or Steering" the major increase in interval occurs between 
positions 2 and 3, and between 4 and 5, with a decrease between positions 
5 and 6. (Compare column (5) with column (8) J An examination of the 
descriptors [2, Part B, Vol. Ill, p. 2-20] suggests that the differences 
relate to degree of precision and the distinction between small number of 
stimuli and extremely complex external arrays of stimuli ♦ The scaled dif- 
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Table 1. DATA BASE FO!: SCALE V AL>:^S USTNC THI3R ST0NE EQUAL lOTERVAL SCALING 
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o 



t 



Sral p Name -NiimbeT" I 

4J V> O JL. w 41 0 tU w • ^1 Villi ^ % « 
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d i e 


f • R 
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1* Freauencv 


Median bcaJe value" 
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0 3* 


3 0* 


9.0*i 1.0* 
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4.0* 




(15) 


Interquartile Range 


1.0 0,9 


0.8 " 


n s 


1 1 
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0.1 


























Median Sccla \<iluc 
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K9 0-6 
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1.0 


1 q 
































Median Scale va^ue 


1.5'-' Q.W- 


S.O'^ 


9.0* 




3.5*: 7.5* 


3.0/ 






fion (15) 


Interquartile Raage 


l.O 0.5 


O.o"" 


0,6 


■ 


1.2 • 1.0 


0.') 


























Median Scalr- y^alue^ , 


1.5* A.O/ 


9.0* 


7.0* 


0.0* 


r,.0*t 5.5*1 8.0/ 






-fno (15) 


Interquai tile Range 


r.l 1.2 


0.4 


^1.0 


0.5 1 


T4 1 1.0 I 0.9 










r 






I ' A 






5. Human Interaction 


~ 

Median Scale Valvu?^ 


5.0* 7.0* 


3.0* 


0.0* 


9.0* 


l.O*' 
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Interquartile Kan^:,e 


1.1 1..0 
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0.0 
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4.5// 4.0/ 


1,0#I 8.5." 


5.5/ 


3.5/: 


6.5// 


5.5/ 


3.0// 
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1.5 2.0 


0.7^;2.1 


3,1 


1.4 1 


2.1 


1.6 i 1.2 

! 


0.4 












i i 
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7. Oral use of 


Mediar Scale Value^ 


' 4.0* 7.5* 




0.0* 


9 0* 


' 




1 





Language (18) 


Interc{uarril^ Range 


1.2 1.1 


1.0 


0.5 ] 0.6 


1 

, . * 
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j 




8. Reading Use of 


Meuian Scale Voluc^ 


' 2.0* 7.0/ 


0.0* 


9.0 


5.0* 






4 




Language (17) 


Interquartile Range 


I 1.2 0,9 


0.5 


o.« 


1 6 
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9. Written Use of 


Median Scale Value^ 


6.5*' 5.0* 


9.0* 


2.0* 


CO* 










Language (15) 


Interquartile Rangr j IJ. | 1.1 


0.8 


0.8 


0.5 











* Item was kept. 

// Item was edited. 

/ Item was eliminated. 



^ Rounded 

b Refers to r *:)er of judges in equal interval test. 




Table 2. NEW SCALE VALUES USING INTERQUARTILE RANGES 
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1 
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1.5 


0.0 


0.0 


1.5 


0.0 1 
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1.0 


1.0 


1.9 


2.9 


1.5 


1.5 


3.5 


0.0 : 


a 


1.9 


1.9 


0.9 


2.8 


4.4 


5.0 


2.0 


0.6 
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0.9 


0.9 


0.6 


1.5 


7.2 


7.0 


2.0 


0.2 


b 


0.6 








8.7 


9.0 




0.3 ; 






Guiding or 


Steering 


Skill Scale 






e 


0.5 


0.5 


1.1 


1.6 


0.0 


0.0 


1.5 


0.0 


a 


1.1 


1.1 


1.4 


2.5 


1.6 


1.5 


1.5 


0.1 


f 


1.4 


1.4 


1.0 


2.4 


4.1 


3.0 


2.5 


1.1 


g 


1.0 


1.0 


1.0 


2.0 


6.5 


5.5 


1.5 


1.0 


d 


1.0 


1.0 


0.4 


1.4 


8.5 


7.0 


2.0 


1.5 


c 


0.4 








9.9 


9.0 




0.8 



Note: From values like these, a linear transformation may be made to 

produce equal midpoints and ranges for all scales; this, of course, 
throws away some useful data regarding differences between scales. 



ference in the new scale values is somewhat larger than in the procedure 
followed in the study. It is possible that the smaller increment between 
the last two positions in the initial scaling is due to "end effect," 
which would tend to suppress the median value more than the standard de- 
viation. 



For "Locomotion," in contrast, a slight shortening of the scale 
seems to occur, mostly between the second and third descriptors, but this 
is compensated by the increase between the third and fourth. From the 
descriptor content (2, Part B, Vol. Ill, p. 2-18] the issue is the place- 



G-39 

311 



ment of "moderate" between "low" and "high" as adjectives used in the de- 
scriptors with regard to degree of body coordination. The differences 
are interesting, from a psychometric point of view, but it is doubtful 
whether the new scale values would make any major change at the level of 
correlation or factor loading. If new scales are developed, however, it 
should be kept in mind that the proposed method of equal-interval scaling 
uses much more of the relevant data and, given current computing technol- 
ogy, would take no more effort than the simpler method of "equal-appear- 
ing" intervals. 

The HSMS indexes used to assess the scale-s for reliability and 
validity are logical, although strange to me. It would be helpful to have 
a reference to their derivation. or , if they are originaKto show their rela- 
rion to Lambda or the Contingency Coefficient. Considering that the rat- 
ings are ordinal, if not completely equal-interval, the opportunity ex- 
ists to use an analysis-of-variance model for reliability and also for 
content validity (accuracy). As reliability coefficients are notorious 
for instability across samples and situations, it would be prudent to 
check routinely for each job analyzed. 

There is a confusion in a footnote of the document reporting on 
the pilot test^ between variance, in the usual sense related to range, and 



Footnote reads as follows: "The six-factor solution accounts for 73 per- 
cent of the variance. (Variance refers to a statistical measure which 
reflects the different scale values of each of the variables as found in 
the ^asks. The greater the range and distribution for the scales in the 
task'data, the greater the variance.) The fact that the large number of 
variables are accounted for by a small number of factors with as much as 
73 percent of the variance accounted for is considered statistically very 
satisfactory." [3, p. 3-7]. ^^^^ 
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variance In the factor-analytic sense, which really means covariance. 
Each factor can be used to reproduce a partial correlation matrix; in 
this case, one could obtain six partial-r-matrices. If these were added 
cell by cell, the resulting sum would closely approximate the correlation 
matrix between the variables initially computed from the task-by-SK ma- 
trix. The amount of "variance" in each of the partial matrices is the 
sum of squares of the loadings of variables on that factor. The total 
sum, called the trace, is equal to the number of variables when principal 
components are used; otherwise it is the sum of the communalities of the 
variables. The proportion of variance is thus the trace of the partial 
matrix divided by the trace of the initial matrix. 

Davis' [1] discussion of robustness can be considered reassuring, 
given that sample size is interpreted as degrees of freedom. In the HSMS test 
data, degrees of freedom, computed as the number of tasks (replications) 
less 2, less number of variables, less number of derived parameters (fac- 
tors) was 121. One can be reasonably confident about the eigenvalues here, 
but somewhat less so about the specific values of loadings. In my reading 
of the psychometric literature, the lowest ratio of replicates to variables 
I have come across is 5 to 1, a good bit larger than in the present study. 
However, it is doubtful that the clusters, in the large, would differ much 
if the number of replications were increased or the number of variables 
decreased, if that were done proportional to the present number and rep- 
resentativeness of variables and tasks. 

On the other hand, were one to increase the number of tasks as 
noted earlier, to reflect the relative frequency of tasks across jobs, and 
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if one were to reduce the number of knowledge categories by consolidating 
rationally^ at some more general level of description, ciy the level of 
5 or 6 digits in the coding system, then one could anticipate greater con- 
fidence in the results, with perhaps a few differences which would be in- 
teresting. Again, I hasten to say thau I am suggesting a trade-off, and 
not making a specific criticism. The results, even though from a single 
institution, are—to say the least— exciting and hope for the applicability 
of the method in other similar studies is well warranted. 

I must note in passing th?*: the factor analysis of dichotomous 
data has long been practiced and studied in the psychological literature. 
Phi is, after all, a product-moment coefficient and, if the dichotomies 
are made near the medians of the distributions, it is no great effort 
to presume an underlying bivariate normality for each pair of variables. 

It is discouraging to note that the "loadings" of tasks on fac- 
tors are not comparable from one factor to another, as noted in footnote 
c. Figure 16.^ This situation would not arise with factor scores. 

There is a frequent error of interpretation of factor loadings- 
frequent among many practicioners in the field— to which this study seems 
to fall prey. This error is to interoret a loading as one would a mean, 
e.g., to infer that a high loading of an SK variable on a factor implies 



^ Footnote reads as follows: "Loadings represent the degree to which task 
partakes of factor. Loadings are not standardized, and sign has no in- 
trinsic meaning except for change from high on one, passing through zero, 
to opposite sign, as continuous hierarchy." [3, p. 3-31]. 
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that a high degree of what the variable refers to Is required, In some 
absolute sense. Not so: loadings are based on correlations, which are 
based essentially on rank order similarities among variables. I.e., the 
variables that are correlated differentiate among the tasks In the same 
way. Relative standing is all one gets from correlations (one can get 
closer to absolute values In the analysis of cross-products), and It Is 
necessary to go beyond the loadings to factor scores to see anything about 
actual levels on whatever scale Is being used. In the present study, it 
is likely that the second-mode values, relating tasks to factors, are r40t 
too different from factor scores and that their Interpretation is appro- 
priately made. However, the specification that correlations wete used, 
and the lack of sufficient detail about the two-mode computations (were 
correlations used to compute the mode 2 factors as well as mode 1?) leaves 
a residual ambiguity that is nettling, but not sufficiently disturbing to 
rouse a strong criticism. Again, it is difficult to argue with the re- 
sults . 



has a high degree of Involvement with all tasks, but its variance is small 
relative to error of measurement (lack of reliability) , then it will prob- 
ably have low correlations with other variables, and thus low loadings on 
all factors. Similarly, a variable with small Involvement with all varl- 
lables will have low loadings on all factors. Obviously, given only that 
a variable has low loadings on factors, one cannot tell which is the case 
with regard to Involvement level, but only that there is little differ- 
ence in level of Involvement from one task to another. 



Here is an example of the interpretation problem. If a variable 





To look at the question of proficiency, one must have two in- 



puts: factor loading of variable, and score of task on variables. In gen- 
eral, if a task has the same score on two variables, its relation to the 
f actor (s) involved depends on the size of the loading of the variable(s) 
on the factor(s). If two variables have the same value for loading, then 
the task-factor relation depends on the differences in the task-variable 
score. A positive factor score means only that both loading and score on 
variable had the same sign (+ or -) ; a negative factor score means only 
that the sign of the loading differed from the sign of the task on vari- 
able score. The sign situation is further complicated by the factor rota- 
tion rule which, essentially, assigns a + or - sign to loadings on a fac- 
tor depending upon whether the last rotation was to the left or to the 
right. Because this direction does not affect the structure, in that the 
correlation matrix can be reproduced with either sign for a factor, it is 
permissible to change all signs of all variables on a single factor; +'s 
become -'s and -'s become +*s. This change is often convenient when one 
wishes to interpret factors as ways of differentiating observations, or, 
in HSMS context, when one wants to describe SK factors as ways of differ- 
entiating tasks. When one wishes to interpret factor scores, or "loadings" 
from two-mode analysis, corresponding changes may be made. With factor 
scores, the change is direct, simply the changing of signs of factor 
scores for all factors for which the variable signs were changed. With 
two-mode analysis, because of the inverse transformation, it is not so 
apparent that a simple change will suffice. This point needs empirical 
study. 



ERIC 




To comment directly on the data, it appears to me that the task 
hierarchies developed for the factors are reasonable [3, Figure 16, p. 
3-31], in that certain groups of tasks seem related to certain skills and 
knowledge groups. Whether this is the optimal definition of a "job" may 
be another question. Again the question of comparability of casks across 
factors becomes critical for interpretation of the total array. 

Despite the issues raised above, 1 am struck by the consistency 
and apparent substantive coherence of the results , especially the task hier- 
archies. Wliat might be revealed when the few obscurations are removed? 

SUMMARY 

I sense that there is a great effort underway, for which sound 
strategy at the highest levels has been laid out. My major concern is 
that the subordinate decisions, and the execution of the strategy, partic- 
ularly in some of the more elaborate statistical procedures, seems to hav 
proceeded by fiat, for convenience. On the other h^^nd, when "it works," 
a method certainly deserves further application. 

In my critique, I pointed to three trade-offs: 

1. Inclusion of tasks on a more representative basis rather 
than just once, at the cost of using factor scores in- 
stead of two-mode factor analysis. 

2. Considering oblique rotations rather than orthogonal, at 
little cost but with the implicar.ion that the goal is to 
maximize rather than minimize lateral relations among 
tasks across jobs. 

3. Reducing the number of SK variables through a rational 
consolidation, losing some specificity but gaining greater 
confidence in the stability of the statistical estimates, 
due to a larger ratio of replicates to parameters. 
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In addition, I suggested ways in which the scaling might be made more 
precise and the relation of jobs to skills and knowledge made more formal, 
using a multiple-regression model. 

Technology moves rapidly, and it must be said that the decisions, 
made some five to seven years ago, were consistent with the technology as 
it was then. Current options allow more refined approaches to these prob- 
lems, but no basic change in strategy is indicated. 

Finally, I would be remiss not to report my feelings of excite-- 
ment and challenge at this new vista in job analysis; my comments are to 
be interpreted as my attempt to make a really good thing a little better. 
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A CRITICAL REVIEW OF THE METHODOLOGY AND STATISTICAL TREATMENT 

OF DATA IN THE TASK ANALYSIS AND CAREER LADDER DESIGN OF 

THE HEALTH SERVICES MOBILITY STUDY^ 

2 

By Mark I* Appelbaum 



INTRODUCTORY COMMENTS 



Any critical review of statistical and methodological orocedures 
must proceed from the context in which the techniques were applied. Sta- 
tistical and methodological techniques are rarely in and of themselves 
either correct or incorrect, but rather depend upon their particular usage. 
This fact, which is commonly recognized in the physical sciences (e.g., 
the differential methods of chemistry vs. chemical engineering), is often 
ignored in the behavioral and social domain where arguments abound con- 
cerning the appropriateness of certain techniques in absolute terms. 

It is, perhaps, unfortunate that workers in the social sciences 
have failed to recognize the distinction between pure and applied research 
and hence have used the same critical standards for both classes of work 
without regard to the appropriateness of such a decision. 



1 This document was completed by Dr. Appelbaum in July, 1976, and is in re- 
sponse to a request for review of the Health Services Mobility Study (HSMS) 
method. Documents supplied for review purposes included the Davis paper 
[2] (which precedes this), listings of the HSMS computer programs requested 
by the author, early HSMS documents not currently available [3], Research 
Report Numbers 4 and 5 [4], Technical Report No. 13 [5], and current HSMS 
scales not yet published. Dr. Appelbaum discussed aspects of the method- 
ology with Eleanor Gilpatrick, Director of HSMS. Numbers in brackets refer 
to references listed at the end of this paper. 

^ Dr. Appelbaum is Associate Professor of Psychology and Associate Dean of 
the Graduate School, University of North Carolina at Chapel Hill. He is 
also involved in research on National Assessment of Educational Progress, 
and a consultant for National Science Foundation and Behavioral Technology 
Consultants. _ . - 
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It is often the case, however, that the highly precise methods of the 
laboratory are too finely tuned for the applied piece of research, just 
as the highly sensitive torsion balance of the chemist is an inappro- 
priate instrument for the chemical engineer. Conversely, the cruder meth- 
ods which might be appropriate for applied research are not necessarily 
sensitive enough for tasks of the pure researcher. It is therefore manda- 
tory that the research instruments be •'udged in the Context of their use. 

SCOPE AND ORGANIZATION OF THE HEALTH SERVICES MOBILITY STUDY 

At the onset it must be noted that the scope and purpose of 
the Health Services Mobility Study (HSMS) is nothing short of monumental. 
Even with the limited classes or areas included, the task of organizing, 
measuring, and interrelating the many specific tasks, skills, and knowl- 
edge requirements is a vast undertaking. Particularly is this the case 
in light of the very limited set of techniques available for evaluation 
research and the even smaller number of well executed studies to serve 
as models. Taken as a whole I find t^e technical and methodological por- 
tions of the study indeed well conceived and executed. 

EVALUATION OF THE METHODOLOGICAL COMPONENTS OF THE STUDY 

There are three major aspects of the study which afford the 
possibility of a methodological evaluation. These are (1) the initial 
selection and organization of the task, skill, and knowledge components; 
(2) the measurement of these components; and, finally, (3) the interre- 
lating of the components in order to identify a simplifying organization 
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and to allow the development of job and curriculum ladders. The first 
of these is outside of the area of competency of this reviewer and shall, 
consequently, be left untouched. 

Measurement of the Various Task, Skill and Knowledge Components 

In the present study the process of measurement is fundamentally 
one of scale construction. The methods employed to develop the various 
scales are, perhaps, the most extensively documented in the entire report. 
In general, well known and highly accepted classical methods were em- 
ployed — namely, constructions using equal interval methods. The quality 
of the resulting scales (and their consequent utility) depends, of course, 
upon the ability of the judges to construct truly equal interval scales. 
Being unfamiliar with the judges or their degree of training, it is diffi- 
cult to assess the degree to which they were able to accomplish this task. 
Alternative methods of scale construction do not seem to be applicable 
in this study — the technical requirements being untenable (e.g., n.ethods 
based upon paired comparisons) . 

In terms of the reliability of the scales, I was unfamiliar 
with the methods employed, but saw nothing which appeared to be, on the 
surface, inappropriate. 

Interrelating of the Components In Order To Identify a Simplifying 
Organization 

This section deals largely with the utilization of the "Two Mode*' 
Factor Analysis procedure and issues attendant thereto. Discussion of the 
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Davis paper [2] (which appears in this document] is also included, 

••Two Mode'^ Factor Analysis 

Among the difficulties involved in understanding the "Two 
Mode^' approach, is that (a) there is little existing documentation of 
the approach and (b) the bulk of the procedure is defined by the pro- 
gram which is used to produce the results • Perhaps a few words about 
the procedure as applied to the Health Services Mobility Study would be 
of some help. It is first necessary to understand that the basic in- 
put data are unlike those usually employed in factor analytic studies, 
father, what is employed is a matrix which has as its rows the various 
tasks (T) and as its columns the various skill/knowledge measures (S). 
The data entries are the judgments (actually a single "average" or 
••consensus" judgment) as to "how much" of a particular skill or knowl- 
edge is required for a particular task. The judgments are (as de- 
scribed in several of the reports) the expert ratings of trained judges 
for a rather idealized task, but one which corresponds to actually occur- 
ring tasks. 

From this single basic data matrix two conceptually different 
but necessarily related "correlation" matrices are formed; one is ob- 
tained by intercorrelating tasks over the various skill/knowledge mea- 
sures, the other by intercorrelating the skill/knowledge measures over 
tasks. (In the more usual application of this procedure, usually referred 
to as a Tucker-Messick Points of View Analysis, these would correspond to 
the correlation of individuals over variables (also referred to by 
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Cattell [1] as the Q technique), and the usual correlation of variables 
over individuals (Cattel's R technique [1]). 

These matrices can be thought of as representing tasks in a 
skills /knowledge space and skills/knowledge in a task space. By using 
the results of the well known Eckert-Young theorem it is possible to 
approxiruate tne original rating matrix as the product of two concep- 
tually different sets of relations based upon these, two spaces. Specif- 
ically, Eckert-Young states: 

n m n^k k km 

where ^^X^^^ is the original matrix of ratings; j^Uj^ are the k 

characteristic vectors of X'X (the skills/knowledge measures 

correlated over tasks); ,V' are the k characteristic vectors 

km 

of XX' (the tasks intercorrelated over skills /knowledge mea- 
sures) , and A is the common diagonal matrix of the character- 
istic roots of X'X and XX' which must be identical* 

The first phase of the "Two Mode" program is to then extract from the 

X matrix, the three matrices , A , and t.V'. 
n m n K fv m 

Having these three "basic" working matrices and noting that 
n^k ^ n\^ principal components of the skills/knowledge corre- 

lation matrix, one may then proceed to find a particular representation, 
there being infinitely many with respect to the reproduction of ^X^ 
(the factorial invariance problem). Thus one may choose, for instance, 
to use the varimax representation of the principal components of the 
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skills/knowledge correlation matrix as a starting point for a "nice" 
representation . 

Let T represent the kxk non-singular transformation which 
achieves the variraax rotation, (We eliminate subscripts in this presen- 
tation.) Thus: 

R « PT 

where R is nhe variraax rotated solution, (T is analytically 
determined by a varimax program.) Sinct the Varimax Trans- 
formation Matrix is non-singular, T"^ exists. 

We may now write: 

X = U X V (Eckert-Young) 

« PV* (definition of principal components) 

= PTT"''-V' (since TT"^ « I) 
= (PT)(T"''-V') 

= R(T"'^V') where R is the varjmax representation. 

Thus, if we wish to use the varimax representation of the 
principal components of the skills/knowledge correlation ma^rix, we 
must also transfer the principal components of the task correlation ma- 
trix in order for the equality to hold. As can be easily seen abovfi, 
the required transformation is T^^ (this is the so-called, and mysteri- 
ous, counter-rotation). It is the second phase of the program which 
achieves this rotation and "counter rotation." 
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The PCVARIM and "Two-Mode" Programs 

The complete computer programs as supplied by HSMS were re- 
viewed by visual inspection. While it would have been preferable to 
actually make the programs operational and ran test data on them, this 
option was judged impractical in terms of the time and personnel involved. 

Much of the "Two Mode" program listing is simply system 
overlay and input/output routines which do not concern us in this evalu- 
ation. The functional portion of the program is actually a fairly 
standard common factor analysis program (or principal components pro- 
gram, depending on how the communalities problem is handled). The 
major computational routine, the eigenvalue/eigenvector routine, is the 
rather dated and out-of-fashion "HOW" routine using tridiagonalization. 
While this routine is known to produce certain problems under rather un- 
usual conditions (linearly independent but nonorthogonal vectors when 
there are roots of multiplicity greater than one), it is highly un- 
likely that this situation would occur in this particular application. 

The second mode of operation, the rotation and counter-rota- 

l 

tion of the basic components matrix, while unusual in factor analysis 
programs, is based upon well known methods of rotation and should cause 
no unusual problems. This method, of course, offers no solution to the 
basic invar iance of factor score problem — but, on the other hand, adds 
no additional problems which cannot be said of many other routines. 

In general, one can have reasonable confidence in the numerical 
results of the program. Given sufficient time and funds, however, one 
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might wish to introduce the never and less problematic numerical 
solution to the eigen problem* 

Alternative Methodological Strategies 

Given the basic task at hand, one might ask if alternative 
methodologies are available which might rather have been employed* 
Taking as an oversimplified statement that the goal was to organize 
the task-skill components of selected jobs within a particular health 
delivery unit, several alternative methodologies suggest themselves » 
These may be divided into two general classes: (a) factor analytic, 
(b) multidimensional scaling* 

Factor Analytic Techniques 

While there are literally hundreds of variations on the basic 
"common factor analysis," all of these have certain common features 
which make them different from the Principal Components analysis actually 
employed* From the onset the goal of Principal Components was the 
stTT^arization of the total variance in a test space into a smaller number 
of orthogonal components. The goal was simply to summarize* All of the 
factor analytic techniques, however, have a goal of first reducing the 
total variability into a smaller "common space" variability (the commu- 
nality problem) and to then find a representation in that reduced space 
which maximally reproduces the intercorrelations among the manifest vari- 
ables* Many of the problems currently discussed by factor analysts center 
around thcs** ^^roblems* 
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In terms of the goals of the HSMS project, the Principal 
Components approach seems, to this reviewer, the more straightforward 
approach and the one that most nearly will achieve the desired end pro- 
duct. It might also be mentioned that Principal Components are far 
easier to work with and interpret. Save the problem of whether to 
analyze cross-products, variance-covariance, or correlations, the 
results are completely determinate—Principal Components being the so- 
lution to a statistical maximization problem. While it would not be 
valid to select a method based simply upon ease, given the fulfillment 
of the basic project goals by Principal Components it seems a wise se- 
lection. 

Multidimensional Scaling 

Major advances have recently been made which allow consider- 
able flexibility in creating spatial mappings of both stimuli and indi- 
vidual "points of view." The basic data requirements of this approach 
are, however, such that it would seem inappropriate to consider this 
methodology. T:. use this method it would be necessary to have paired 
judgments of "task similarity" for all tasks in all jobs. This would 
require judges who had familiarity with all jobs and all tasks within 
jobs as well as the ability to assess their similarity. Given the 
breadth of the jobs included and their complexity, it seems highly un- 
reasonable that such judges would be available. 
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Coimnents on the Davis Paper [2] 

The presentation given in the Davis paper [2] (presented 
herein) gives an adequate but not detailed review of the Two-Mode Factor 
procedure — one not well documented elsewhere. It would be helpful if 
more detail were added. It should be emphasized, however, that the 
unique portion of the technique is not so much a part of the classical 
factor problem as one of representation (actually related to several 
classical problems; namely those of factor scores and congruences). 
While apparently appropriate for the Health Services Mobility Study, 
the technique should not be used, in general, without great care and 
understanding. 

There are a few issues raised which deserve some comments, al- 
though they have little direct impact on the Health Services Mobility 
Study. The first of these concerns is the selection of the correlation 
matrices as the unit of analysis. While this discussion is rather casu- 
ally presented in the Davis paper, and while the decision is, most 
probably, the correct one, it is still worth noting that there is a 
rational decision. Since the scales which are being inter-correlated 
are themselves rather arbitrarily constructed scales (in the ^^ense that 
adding a constant or using a constant multiplier would do no violence 

< 

to the scales per se), one would not wish the resulting solution to be 
dependent upon the means or standard deviations of the scales. Had the 
cross-products matrices been used, the solution would have been depenc'ent 
upon both the means and standard deviations; had the variance-covariance 
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matrices been used, the solution would have been dependent upon the 
standard deviations. Since neither of these were desired, the choice 
of the correlation matrix was indeed appropriate — if not defended. 

The second issue of concern deals with the rather oblique 
treatment of factor analysis versus principal components. While it 
might be of some comfort to note that these two conceptually different 
techniques have similar results, I do not find it pleasing to base the 
"justification** of using principal components on that fact. It seems 
to me that principal components, on a purely theoretical level, does 
exactly what was desired and should thus be the analysis of choice on 
that basis exclusively. The goal was to summarize the total variation 
in the system; there was no concept of common space variance. On that 
basis alone one is perfectly justified in using components — no additional 
justification is needed. 

The final comment pertains to oblique versus orthogonal repre- 
sentation. To this, two comments are relevant. First, principal com- 
ponents art, by definition, orthogonal. Second, a representational basis 
is completely a matter of taste. 

SUMMARY 

Given the orientation stated in the Introductory Comments, it 
is the belief of this reviewer that, hut for a very few technical points, 
the basic methodological approach taken in the Health Services Mobility 
Study is sound. While other investigators may have chosen other approaches, 
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there are no problems, beyond those of personal taste, which would 
force the conclusion that some other approach should be preferred. 
It is particularly important to note that when dealing with an area so 
complex as this, tue most one can reasonably hope for is that a few 
basic and important findings should result* Certainly this has been 
achieved • It is perhaps equally important to note that there are years 
of additional work (and large sums of additional funds) required before 
anything approaching completeness could be hoped for. 
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